Python Tutorial: Expanding ECLIPSE DATA file

In this post, I am going to walk you through an old code that I wrote a couple of years ago. For reservoir engineers ECLIPSE numerical simulator is synonymous to Simulation.

ECLIPSE (ECL) has a lot of advantages over its competitor that made it industry standard. One simple advantage was the ability to use ‘INCLUDE’ files. INCLUDE files simply allow you to reference the input data in the main file, while your actual input can reside in another location. This prevents the main ‘deck’ from getting too busy.

That is great on a day-to-day operation, but sometimes you prefer to have all the data in one single file, particularly during troubleshooting an ECLIPSE error. $EXPAND macro that comes with ECLIPSE installation allows you to bring all the INCLUDES input data in to the main ‘deck’.

One day an admin right issue access meant I could not use the $EXPAND or any other ECL macro anymore, and as I was on my path to develop my Python skills, I tookthat as an opportunity(excuse) for development. I was also excited as the solution seemed benefited from implementing recursive functions.

Let’s briefly review what a recursive function is before reviewing the code. You should be already familiar with a function in the context of programmnig is. A recursive function is special since it can calls itself within its own definition. In other words, a function that refers to itself in its own body.

When a recursive function is called, it breaks down the problem into smaller sub-problems and solves each of them recursively until a base case is reached. The base case is the condition in which the function does not call itself, and the recursion stops.

For example, let’s consider the factorial function, which is commonly defined recursively. The factorial of a non-negative integer n is the product of all positive integers less than or equal to n. The recursive definition of the factorial function can be written as follows:

def factorial(n): 
  if n==0: 
    return 1 
  else:
    return n*factorial(n-1)

In this definition, the base case is when n=0, and the function simply returns 1. For all other values of n, the function calls itself with the argument n-1 and multiplies the result by n.

Recursive functions can be a powerful tool for solving problems that can be broken down into smaller sub-problems. However, it’s important to ensure that the recursive function does eventually reach a base case, or else it will continue to call itself indefinitely and cause a stack overflow error.

Now let’s review the code to expand the whole DATA file and bringing all the include files information to the main deck. You can see the complete code in my GitHub repo, here.

First, let us initialise the code:

import os 

file_name = 'DATA file name goes here' 
path = r'The path to the folder containing DATA file goes here' 
final_file = []

In this section, we import os library and give option to user to provide file_name, and path to the file.

Next step is not mandatory, but a nice section to have. Here the code checks if the file extension is *.DATA or if the user has provided a file with different extension. We are going to assume the file name is the same as the original DATA file, hence, changing the extension to match. This is a good practice to make the core more robust for different scenarios.

At the end of this block, file contains the full path to our *.DATA file.

if file_name[-5:] == '.DATA': 
  pass
else: 
  ext_ind = file_name.find('.') # Making sure the DATA file has the correct extension if ext_ind > 0: file_name = file_name[:ext_ind] + '.DATA' else: file_name = file_name + '.DATA' file = os.path.join(path, file_name)

Now we can create our recursive function as below:

def expander(data_file): 
  flag = 0 #flags where code detects include files 
  grid_flag = 0 # if grid files are needed set this number to non-zero 
  
  include = os.path.join(path, data_file)
   
  with open(include, 'r') as inc:
    for item in inc:
      if item[:7] == 'INCLUDE': 
        flag = 1 
        continue
      elif flag == 1 and grid_flag != 1 and item[:2] != "--" and item[0] != '\n':
        loc = item.find('/') # finding the path to the include
        item = item[:loc].strip() # removing spaces and new lines from the path
        item = item.strip("'") # removing the qoutations from the path name 
        expander(item) # recursive function if there is other includes within this include 
        flag = 0 # flag back to zero so the lines can be writtern now
        continue

      elif item[:5] in ['COORD', 'ZCORN'] and grid_flag == 0:
        grid_flag = 1
        continue

      elif item.strip() == "/" and grid_flag == 1:
        grid_flag = 0
        continue

      elif grid_flag != 1:
        final_file.append(item)

return final_file

The code has enough comment that you should be able to follow. but the idea here is we scan the *.DATA file, if we see any include file, we also scan that and attach it to the block of text in memory, with the exception of GRID section (optional) to reduce the volume of the text in the final file. If our include files have their own include files (mostly common in SCHEDULE section), our recursive function will call itself out and read them too.

Now that our function can read the file, we just need to simply, call our function and write it to a destination text file:

with open(os.path.join(path, file_name[:-5]) + '_EXPANDED.DATA', 'w+') as output:
  output.writelines(expander(file_name))

Finally, we can tell the user, the operation has been successful and where they should be able to find the expanded DATA file:

print('You can check the expanded DATA file in the following path: ')
print(os.path.join(path, file_name[:-5]) + '_EXPANDED.DATA')

I hope you enjoyed this tutorial. Feel free to drop me an email on hello@aminnoor.blog, or comment below if you have comments or questions.

Originally published at https://www.aminnoor.blog.