DownUnder Geosolutions: HPC Optimizes Energy Exploration for Oil and Gas Startups
In its quest to meet the world’s ever-increasing demand for energy, the oil and gas industry has become one of the largest users—and leading innovators—of high performance computing. As natural resources deplete, and the cost of accessing them increases, highly sophisticated computational modeling becomes an essential tool in energy exploration and development.
Advanced computational techniques provide a high-fidelity model of the subsurface, which gives oil and gas companies a greater understanding of the geophysics of the region they propose to explore. A clearer picture of the earth enables targeted drilling, reduced acquisition costs, and minimal environmental impact. In an industry where time is money, powerful and sophisticated computational modeling provides a competitive advantage.
Finding economically viable reserves is akin to finding a needle in a haystack. The journey from green-field exploration to first-oil is long and expensive. Energy companies rely heavily on seismic data, and the modeling derived from the data, to provide insight and guidance to the exploration process. When an exploration well in deep water can cost more than $100 million (US), the cost of drilling inaccurately is enormous. High quality and fast processing and analysis of seismic data are paramount to a successful exploration and development program.
A single seismic survey can result in hundreds of terabytes of data and require many months of processing on some of the world’s largest supercomputers. Converting the raw data into useful models of the subsurface requires complex algorithms, highly tuned for the most sophisticated computer hardware. The seismic processing industry has been at the forefront of implementing innovative computational techniques, hardware, and advanced cooling solutions as it continues to demand the most efficient HPC to deliver its service.
Based in Perth, Australia, DownUnder GeoSolutions (DUG) is a leading global geoscience company that provides essential services to energy companies to enable them to accurately locate new and productive sites for hydrocarbon extraction. From its four sites in Perth (Australia), London (UK), Kuala Lumpur (Malaysia), and Houston (USA), DUG offers a wide range of products and integrated services for oil and gas exploration and production. To achieve this, DUG requires enormous compute power for processing and imaging seismic data and generating high-resolution models of the subsurface.
Growing from humble beginnings in 2003, in a shed in the founding partner’s backyard, DUG has grown to become a major service provider for the oil and gas industry. Investing heavily in research, development, and advanced technologies has enabled DUG to undertake some of the largest processing projects in the industry.
The process of developing a high-quality, three-dimensional (3D) model of the Earth’s subsurface is extremely complex. First, a seismic survey is performed, which records the reflected sound waves from hundreds of thousands of sound impulses made on the Earth’s surface. This data is then processed to remove various types of coherent and random noise, leaving just the primary signal of interest. The reflectors in the data are then spatially located using an earth model, into a 3D volume, producing images of the reflectors in the subsurface. From this data set, the earth model can be updated and the imaging performed again. Many iterations of refinement lead to a very accurate earth model and correct placement of the reflecting geological layers beneath the Earth’s surface.
A processing cycle such as this can take many months to complete. And the compute requirements for a geophysics company continue to increase, with survey areas constantly growing, and with increasing shots and receiver densities. These factors have led to a data explosion, where a project’s input data is measured in hundreds of terabytes.
Simultaneously, processing is using increasingly complex computational methods, which produce more accurate images and earth models at the expense of floating point operations per second (FLOPS). Upgrading to a more accurate computational algorithm can lead to an orders-of-magnitude increase in the computational effort required.
Couple the increased computational requirements with decreasing project timelines and the result is an environment where fast, powerful, efficient HPC is crucial.
DUG adopted and implemented massive clusters based on the Intel Xeon Phi 5110. co-processor. While this computing environment was sufficient to meet current demand, it did not have the capacity for the company’s growing customer base and new technologies.
DUG wanted a flexible and powerful HPC system that could handle large-scale seismic processing and imaging, address escalating demand for advanced algorithms, increase operational efficiency while reducing total cost of ownership, integrate into the DUG ecosystem, and effectively leverage the years of code and algorithm optimization undertaken.
DUG worked with Intel to develop an innovative, energy-efficient solution capable of handling the demanding workloads and escalating compute demands. DUG complemented its existing systems with thousands of the latest systems based on the Intel Xeon Phi processor 7210. Each processor has 64 cores and handles up to 256 simultaneous threads for fast, seamless processing. The clusters are homogeneous and all algorithms can run on the same CPUs, so DUG does not need to target different parts of the cluster with different algorithms. The homogeneous cluster also enables DUG to optimize and prioritize workload scheduling and to process a single job across the entire cluster for faster results.
DUG upgraded its Houston supercomputer, nicknamed “Bubba,” with Intel Xeon Phi processors, doubling its compute power to 12 petaflops single-precision. The company is now upgrading its Houston computer room to give Bubba the ability to grow to a 120-plus PF machine by 2018. DUG is upgrading its other supercomputers, “Bruce” (Perth), “Bohdi” (Kuala Lumpur), and “Bazza” (London) to coincide with new offices and data centers currently under construction. DUG is moving toward an exascale system based on Intel Xeon Phi processors in 2018.
The new supercomputers enable DUG to reduce run times, increase efficiency, and extend the company’s capabilities. Bubba can now run 12,900 square kilometers of 5-kilometer aperture pre-stack migration, or 7,200 square kilometers of large aperture 3D SRME (3D surface-related multiple elimination) in 24 hours. DUG’s HPC environment can run the latest emerging techniques, such as large-scale full waveform inversion (FWI), which uses the entire seismic wavefield to generate high-resolution velocity models for imaging and characterization. The new machine is able to run 3D FWI up to high frequencies of 60 Hz, for example.
DUG also uses other cutting-edge technologies in its data center. One remarkable innovation is its unique modular cooling system, in which servers are immersed in circulating dielectric fluid, eliminating the need for fans and offering considerable energy savings. Combining the Intel Xeon Phi processor with DUG’s proprietary cooling method has enabled the company to significantly reduce its computer footprint and has reduced power consumption by 40 percent. The running costs of HPC systems are commonly evaluated using the power usage effectiveness (PUE) metric. DUG has achieved a PUE of 1.05, considerably better than the PUE of high-efficiency “green” data centers.
Moving to Intel Xeon Phi 7210 processors in DUG’s Houston, London, Kuala Lumpur, and Perth data centers enables greater supercomputing power in significantly smaller spaces. The improved processors require less peripheral equipment on the motherboard, use less energy, and reduce the space required for housing the equipment. This has allowed DUG to install systems with more than 10 petaflops into less than 100 square meters of floor space in a normal office building.
DUG provides hardware and software to marine geophysical company Polarcus*, fitting out its fleet of seagoing vessels for marine seismic data acquisition. Dr. Stuart Midgley, DUG’s systems architect, notes that space is very limited on each vessel, so the onboard supercomputing system must be extremely powerful for data processing and imaging, yet consume minimal electrical power and occupy a limited footprint.
“Marine seismic acquisition has unique restrictions. We can only have a few hardware racks onboard and are limited in weight, space, power, and cooling,” Midgley explains. “All our seagoing systems utilize Intel Xeon Phi to get the maximum supercomputing power in the smallest space allotted. Previously very limited fast-track processing could be delivered; however with the Intel Xeon Phi onboard we can provide denoised, 3D-demultiple, and imaged products in the time it takes the vessel to get back to port.” DUG hardware and software solutions provide industry-leading onboard processing.
Midgley stressed that DUG’s ongoing investments in the latest HPC technology from Intel is a competitive advantage for the company. “The oil and gas industry faces major challenges every day,” he says. “Our clients know that we can provide better quality data and faster insights than our competitors. Even in a downturned market we are expanding and taking on the most challenging projects in the industry.”
According to Midgley, Intel technologies dramatically reduce the time required to accomplish the task. “With Intel’s support, every optimization we do for our systems running the Intel Xeon Phi processor 7200 increases performance of DUG’s technology,” he points out.
Many other innovative HPC technologies from Intel such as Intel Solid State Drives and its Ethernet networking have improved DUG’s data quality, accuracy, and analysis speed. As a result, DUG’s supercomputing systems run faster, more reliably, and have improved energy efficiency. The new processors provide more FLOPS-per-watt by utilizing less electricity, while simultaneously delivering much faster computing. Enterprise-grade equipment means data integrity with full data checksumming, from DRAM all the way to bytes on spinning platters.
Midgley explains that Intel technology has helped DUG meet its large-scale challenges. “For one very large project, the surveyed area covered 24,000 square kilometers and had 200 terabytes of raw input data,” he says. “The latest Intel Xeon Phi processors allowed us to accomplish in seven to eight months what previously would have taken years.”
DUG’s research and development team design, code, and optimize the numerical algorithms to take full advantage of the Intel Xeon Phi processors. This has resulted in the shortest time-to-solution in the industry—while utilizing the highest quality algorithms and without compromising data quality. No shortcuts were needed.
Despite the added computing power, the latest CPUs require less energy to perform the computation-heavy tasks. “At our facilities, we have limits on the amount of power we can access,” Midgley says. “Lower energy consumption by the processors means we can get more flops within our energy budget.”
The latest processors and other HPC technologies from Intel enable DUG to process its client’s data faster, more efficiently, and more accurately. The solution is energy efficient and requires less physical space than most supercomputers. DUG clearly sees a positive return on its investment in HPC technology, and, in turn, is able to deliver better services at a lower cost.
Midgley believes the new HPC adds to DUG’s value proposition. “Every advancement in supercomputing performance DUG adopts allows us to obtain higher-quality geophysical data at an even higher fidelity and return more insightful information to our clients,” he says. “This empowers them to make the most informed decisions about new drilling sites and maximizes the potential return from each hydrocarbon reserve. Intel’s investments in HPC technology are central to our business. Every improvement in performance is a competitive advantage for us.