Electrofacies source code depreciated numpy "as_matrix" command
cdmeyer23 opened this issue · 7 comments
cdmeyer23 commented
Stoked to keep working with this program!
I am attempting to run the wolfcamp_single example and I am running into an error:
AttributeError: 'DataFrame' object has no attribute 'as_matrix'
Looking at the Electofacies definition when defining minibatch_input:
components = pd.DataFrame(data = pc.transform(X),
index = df[not_null_rows].index)
minibatch_input = components.as_matrix()
it appears that "df.as_matrix()" has depreciated in a the newer version of pandas. I cannot seem to get the correct command to get the example to run. Could someone help?
Philliec459 commented
You are working on wolfcamp_single.py?
I used this about a year ago and do not remember how I got it working, but on mine I not even see where pandas is loaded?
Best Regards,
…_______________________________
E. Craig Phillips
CEO and Chief Petrophysicist
Crested Butte Petrophysical Consultants
459 Cisneros Lane
Crested Butte, CO 81224
USA
Office: +1 970-343-0730
Mobile: +1 970-343-0730
email: craig@cbpetro.com
Website: www.cbpetro.com
GitHub: https://github.com/Philliec459
The information contained in this electronic message is confidential, it constitutes a profesional and/or industrial secret in terms of the current legislation, and is intended for its recipient only. If you receive this message by mistake or if you are not the recipient thereof, please notify the sender and destroy it.
La informacion contenida en este mensaje de datos es confidencial, constituye un secreto industrial y/o profesional en terminos de la legislacion vigente y se encuentra dirigida exclusivamente al destinatario indicado en dicho mensaje. Si usted recibe esta informacion por error o si usted no es el destinatario del mensaje, favor de notificar al emisor, y destruyalo.
On Jul 9, 2020, at 11:15 AM, cdmeyer23 <notifications@github.com> wrote:
Stoked to keep working with this program!
I am attempting to run the wolfcamp_single example and I am running into an error:
AttributeError: 'DataFrame' object has no attribute 'as_matrix'
Looking at the Electofacies definition when defining minibatch_input:
components = pd.DataFrame(data = pc.transform(X),
index = df[not_null_rows].index)
minibatch_input = components.as_matrix()
it appears that "df.as_matrix()" has depreciated in a the newer version of pandas. I cannot seem to get the correct command to get the example to run. Could someone help?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub <#7>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANSKYPA3YGCDWWCIBJDUV2DR2X3KNANCNFSM4OVZ3LDA>.
cdmeyer23 commented
@Philliec459
Yes, I am working with the wolfcamp_single.py
If you look at the electrofacies source code here:
https://github.com/toddheitmann/PetroPy/blob/master/petropy/electrofacies.py
it requires the import of numpy and pandas.
The example wolfcamp_single.py calls ptr.electrofacies on line 147:
logs = ptr.electrofacies(logs, f, electro_logs, 6,
log_scale = ['RESDEEP_N'])
Philliec459 commented
As I said, I set this up ages ago and do not have all the details.
I definitely see the pandas in this module. Thanks. Let me see if I can retrace my setup and see what I can come up with. I had some issues and solved them, but back then I was not documenting how I did what.
In the plot below I see the Electrofacies, but I have no idea of how I did this.
Did you use setup from Todd’s GitHub?
Best Regards,
…_______________________________
E. Craig Phillips
CEO and Chief Petrophysicist
Crested Butte Petrophysical Consultants
459 Cisneros Lane
Crested Butte, CO 81224
USA
Office: +1 970-343-0730
Mobile: +1 970-343-0730
email: craig@cbpetro.com
Website: www.cbpetro.com
GitHub: https://github.com/Philliec459
The information contained in this electronic message is confidential, it constitutes a profesional and/or industrial secret in terms of the current legislation, and is intended for its recipient only. If you receive this message by mistake or if you are not the recipient thereof, please notify the sender and destroy it.
La informacion contenida en este mensaje de datos es confidencial, constituye un secreto industrial y/o profesional en terminos de la legislacion vigente y se encuentra dirigida exclusivamente al destinatario indicado en dicho mensaje. Si usted recibe esta informacion por error o si usted no es el destinatario del mensaje, favor de notificar al emisor, y destruyalo.
On Jul 9, 2020, at 1:32 PM, cdmeyer23 <notifications@github.com> wrote:
@Philliec459 <https://github.com/Philliec459>
Yes, I am working with the wolfcamp_single.py
If you look at the electrofacies source code here:
https://github.com/toddheitmann/PetroPy/blob/master/petropy/electrofacies.py <https://github.com/toddheitmann/PetroPy/blob/master/petropy/electrofacies.py>
it requires the import of numpy and pandas.
The example wolfcamp_single.py calls ptr.electrofacies on line 147:
logs = ptr.electrofacies(logs, f, electro_logs, 6,
log_scale = ['RESDEEP_N'])
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#7 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANSKYPDYVBR4EJFOQU4BXITR2YLMNANCNFSM4OVZ3LDA>.
Philliec459 commented
I did install Petropy and I see that it resides in my …./Anaconda/Lib/site-packages/petropy subdirectory. It was either a pip or "python install setup.py" type of install that I did back in October of last year. Too long ago to remember the details.
I also see from the emails that they were working in petropy back in April. I have not upgraded and do not know if the changes are affecting you now or not. Below is my electrofacies.py if that is any help. Below that is my wolfcamp_single.py.
I do remember having to make some mods to get it to run, but did not take any notes back then on what I did to get it running. I could have removed some code too???? However, I do have petropy working on my system now. At least on my system it does take a while to run.
electrofacies.py
# -*- coding: utf-8 -*-
"""
Electrofacies is a model to calculate numerical facies from log data.
It uses sckit-learn for standardization and clustering.
"""
import numpy as np
import pandas as pd
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.cluster import MiniBatchKMeans
def electrofacies(logs, formations, curves, n_clusters, log_scale = [],
n_components = 0.85, curve_name = 'FACIES'):
"""
Electrofacies function to group intervals by rock type. Also
referred to as heterogenous rock analysis.
Parameters
----------
logs : list of :class:`ptr.Log` objects
List of Log objects
formations: list of formation names
List of strings containg formation names which should be
previously loaded into Log objects
curves : list of curve names
List of strings containing curve names as inputs in the
electrofacies calculations
n_clusters : int
Number of clusters to group intervals. Number of electrofacies.
log_scale : list of curve names
List of string containing curve names which are preprocessed
on a log scale. For example, deep resistivity separates better
on a log scale, and is graph logarithmically when viewing data
in a log viewer.
n_components : int, float, None or string (default 0.85)
Number of principal components to keep. If value is less than
one, the number of principal components be the number required
to exceed the explained variance.
curve_name : str (default 'FACIES')
Name of the new electrofacies curve.
Returns
-------
list
list of :class:`petropy.Log` objects
Examples
--------
>> # loads sample Wolfcamp calculates electrofacies for that well
>> import petropy as ptr
# reads sample Wolfcamp Log from las file
>> log = ptr.log_data('WFMP')
>> logs = [log]
>> f = ['WFMPA', 'WFMPB', 'WFMPC']
>> c = ['GR_N', 'RESDEEP_N', 'RHOB_N', 'NPHI_N', 'PE_N']
>> scale = ['RESDEEP_N']
>> logs = ptr.electrofacies(logs, f, c, 8, log_scale = scale)
>> import petropy as ptr
# loads logs from a list of paths and
# calculates electrofacies across the wells
#
# defin file_paths for las files to analyze
>> file_paths = ['path/to/log1.las', 'path/to/log2.las',
... 'path/to/log3.las', 'path/to/log4.las']
# create list of Log objects
>> logs = [ptr.Log(x) for x in file_paths]
# define csv with tops for all wells
>> tops_csv = 'path/to/tops.csv'
# add formation tops to wells
>> for log in logs:
... log.tops_from_csv(tops_csv)
# define list of formation tops. If single formation, f = ['FORM']
>> f = ['FORM1', 'FORM2']
# list of curves to use for classification
>> c = ['GR_N', 'RESDEEP_N', 'RHOB_N', 'NPHI_N', 'PE_N']
>> scale = ['RESDEEP_N']
# run electrofacies across logs in list
>> logs = electrofacies(logs, f, c, 8, log_scale = scale)
# save las in renamed file
>> for i, log in enumerate(logs):
... new_file_name = file_paths[i].split('.')[0]+'_with_HRA.las'
... log.write(new_file_name)
"""
df = pd.DataFrame()
for log in logs:
if log.well['UWI'] is None:
raise ValueError('UWI required for log identification.')
log_df = log.df()
log_df['UWI'] = log.well['UWI'].value
log_df['DEPTH_INDEX'] = np.arange(0, len(log[0]))
for formation in formations:
top = log.tops[formation]
bottom = log.next_formation_depth(formation)
depth_index = np.intersect1d(np.where(log[0] >= top)[0],
np.where(log[0] < bottom)[0])
df = df.append(log_df.iloc[depth_index])
for s in log_scale:
df[s] = np.log(df[s])
not_null_rows = pd.notnull(df[curves]).any(axis = 1)
X = StandardScaler().fit_transform(df.loc[not_null_rows, curves])
pc = PCA(n_components = n_components).fit(X)
components = pd.DataFrame(data = pc.transform(X),
index = df[not_null_rows].index)
minibatch_input = components.as_matrix()
components.columns = \
['PC%i' % x for x in range(1, pc.n_components_ + 1)]
components['UWI'] = df.loc[not_null_rows, 'UWI']
components['DEPTH_INDEX'] = df.loc[not_null_rows, 'DEPTH_INDEX']
size = len(components) // 20
if size > 10000:
size = 10000
elif size < 100:
size = 100
df.loc[not_null_rows, curve_name] = \
MiniBatchKMeans(n_clusters = n_clusters,
batch_size = size).fit_predict(minibatch_input)
df.loc[not_null_rows, curve_name] += 1
for log in logs:
uwi = log.well['UWI'].value
for v, vector in enumerate(pc.components_):
v += 1
pc_curve = 'PC%i' % v
### add eigenvector data to header ###
if pc_curve in log.keys():
data = log[pc_curve]
depth_index = components.loc[components.UWI == uwi,
'DEPTH_INDEX']
data[depth_index] = \
np.copy(components.loc[components.UWI == uwi,
pc_curve])
else:
data = np.empty(len(log[0]))
data[:] = np.nan
depth_index = components.loc[components.UWI == uwi,
'DEPTH_INDEX']
data[depth_index] = \
np.copy(components.loc[components.UWI == uwi,
pc_curve])
log.add_curve(pc_curve, np.copy(data),
descr = 'Pricipal Component %i from electrofacies' % v)
if curve_name in log.keys():
data = log[curve_name]
depth_index = df.loc[df.UWI == uwi, 'DEPTH_INDEX']
data[depth_index] = df.loc[df.UWI == uwi, curve_name]
else:
data = np.empty(len(log[0]))
data[:] = np.nan
depth_index = df.loc[df.UWI == uwi, 'DEPTH_INDEX']
data[depth_index] = \
np.copy(df.loc[df.UWI == uwi, curve_name])
log.add_curve(curve_name, np.copy(data),
descr = 'Electrofacies')
return logs
wolfcamp_single.py
"""
==================================
Wolfcamp Example - Single las file
==================================
This example shows the full petrophysical workflow avaiable in PetroPy
for a single wolfcamp las file courtesy of University Lands Texas.
The workflow progresses in these 11 steps
1. Read las file and create a :class:`petropy.Log` object
2. Load tops from a csv file using :meth:`petropy.Log.tops_from_csv`
3. Create a :class:`petropy.LogViewer` show in edit_mode to fix data
4. Define formations for calculations.
5. Calculate fluid properties by
1. Loading parameters via :meth:`petropy.Log.fluid_properties_parameters_from_csv`
2. Calculating over formations via :meth:`petropy.Log.formation_fluid_properties`
6. Calculate mulitmineral properties by
1. Loading parameters via :meth:`petropy.Log.multimineral_parameters_from_csv`
2. Calculating over formations via :meth:`petropy.Log.formation_multimineral_model`
7. Curve summations via :meth:`petropy.Log.summations`
8. Adding pay flags via :meth:`petropy.Log.add_pay_flag`
9. Clustering intervals into Electrofacies via :meth:`petropy.electrofacies`
10. Exporting log statistics via :meth:`petropy.Log.statistics`
11. Saving LogViewer to png and Log to las
To bulk process a folder of las files at once, use the `bulk example`_ .
.. _bulk example: wolfcamp_bulk.html
"""
import petropy as ptr
# import pyplot to add logo to figure
import matplotlib.pyplot as plt
### 1. Read las file
# create a Log object by reading a file path #
las_file_path = '42303347740000.las'
log = ptr.Log(las_file_path)
### 2. load tops ###
tops_file_path = 'tops.csv'
log.tops_from_csv(tops_file_path)
### 3. graphically edit log ###
# use manual mode for fixing borehole washout #
# and other changes requiring redrawing data #
# use bulk shift mode to linearly adjust all #
# curve data #
# close both windows to continue program #
viewer = ptr.LogViewer(log, top = 6950, height = 100)
viewer.show(edit_mode = True)
# overwrite log variable with updated log #
# from LogViewer edits #
log = viewer.log
### 4. define formations ###
f = ['WFMPA', 'WFMPB', 'WFMPC']
### 5. fluid properties ###
# load fluid properties from a csv file #
# since path is not specified, load default #
# csv file included with petropy #
log.fluid_properties_parameters_from_csv()
# calculate fluid properties over defined #
# formations with parameter WFMP from #
# previously loaded csv #
log.formation_fluid_properties(f, parameter = 'WFMP')
### 6. multimineral model ###
# load multimineral parameters from csv file #
# since path is not specified, load default #
# csv file included with petropy #
log.multimineral_parameters_from_csv()
# calculate multiminearl model over defined #
# formations with parameter WFMP from #
# previously loaded csv #
log.formation_multimineral_model(f, parameter = 'WFMP')
### 7. summations ###
# define curves to calculate cumulative values #
c = ['OIP', 'BVH', 'PHIE']
# calculate cumulative values over formations #
log.summations(f, curves = c)
### 8. pay flags ###
# define pay flogs as list of tuples for #
# (curve, value) #
flag_1_gtoe = [('PHIE', 0.03)]
flag_2_gtoe = [('PAY_FLAG_1', 1), ('BVH', 0.02)]
flag_3_gtoe = [('PAY_FLAG_2', 1)]
flag_3_ltoe = [('SW', 0.2)]
# add pay flags over defined formations #
log.add_pay_flag(f, greater_than_or_equal = flag_1_gtoe)
log.add_pay_flag(f, greater_than_or_equal = flag_2_gtoe)
log.add_pay_flag(f, greater_than_or_equal = flag_3_gtoe,
less_than_or_equal = flag_3_ltoe)
### 9. electrofacies ###
# define curves to use in electofaceis module #
electro_logs = ['GR_N', 'RESDEEP_N', 'NPHI_N', 'RHOB_N', 'PE_N']
# make a list of Log objects as input #
logs = [log]
# calculate electrofacies for the defined logs#
# over the specified formations #
# finding 6 clusters of electrofacies #
# with RESDEEP_N logarithmically scaled #
logs = ptr.electrofacies(logs, f, electro_logs, 6,
log_scale = ['RESDEEP_N'])
# unpack log object from returned list #
log = logs[0]
### 10. statistics ###
# define list of curves to find statistics #
stats_curves = ['OIP', 'BVH', 'PHIE', 'SW', 'VCLAY', 'TOC']
# calculate stats over specified formation and#
# save to csv file wfmp_statistics.csv #
# update the line if the well, formation is #
# already included in the csv file #
log.statistics_to_csv('wfmp_statistics.csv', replace = True,
formations = f, curves = stats_curves)
### 11. export data ###
# find way to name well, looking for well name#
# or UWI or API #
if len(log.well['WELL'].value) > 0:
well_name = log.well['WELL'].value
elif len(str(log.well['UWI'].value)) > 0:
well_name = str(log.well['UWI'].value)
elif len(log.well['API'].value) > 0:
well_name = str(log.well['API'].value)
else:
well_name = 'UNKNOWN'
well_name = well_name.replace('.', '')
# scale height of viewer to top and bottom #
# of calculated values #
wfmpa_top = log.tops['WFMPA']
wfmpc_base = log.next_formation_depth('WFMPC')
top = wfmpa_top
height = wfmpc_base - wfmpa_top
# create LogViewer with the default full_oil #
# template included in petropy #
viewer = ptr.LogViewer(log, top = top, height = height,
template_defaults = 'full_oil')
# set viewer to 17x11 inches size for use in #
# PowerPoint or printing to larger paper #
viewer.fig.set_size_inches(17, 11)
# add well_name to title of LogViewer #
viewer.fig.suptitle(well_name, fontweight = 'bold', fontsize = 30)
# add logo to top left corner #
logo_im = plt.imread('cbpetro_logo.png')
logo_ax = viewer.fig.add_axes([0, 0.85, 0.2, 0.2])
logo_ax.imshow(logo_im)
logo_ax.axis('off')
# add text to top right corner #
if len(str(log.well['UWI'].value)) > 0:
label = 'UWI: ' + str(log.well['UWI'].value) + '\n'
elif len(log.well['API'].value) > 0:
label = 'API: ' + str(log.well['API'].value) + '\n'
else:
label = ''
label += 'County: Reagan\nCreated By: Todd Heitmann\n'
label += 'Creation Date: October 23, 2017'
viewer.axes[0].annotate(label, xy = (0.99,0.99),
xycoords = 'figure fraction',
horizontalalignment = 'right',
verticalalignment = 'top',
fontsize = 14)
# save figure and log #
viewer_file_name=r'%s_processed.png' % well_name
las_file_name = r'%s_processed.las' % well_name
viewer.fig.savefig(viewer_file_name)
viewer.log.write(las_file_name)
Best Regards,
…_______________________________
E. Craig Phillips
CEO and Chief Petrophysicist
Crested Butte Petrophysical Consultants
459 Cisneros Lane
Crested Butte, CO 81224
USA
Office: +1 970-343-0730
Mobile: +1 970-343-0730
email: craig@cbpetro.com
Website: www.cbpetro.com
GitHub: https://github.com/Philliec459
The information contained in this electronic message is confidential, it constitutes a profesional and/or industrial secret in terms of the current legislation, and is intended for its recipient only. If you receive this message by mistake or if you are not the recipient thereof, please notify the sender and destroy it.
La informacion contenida en este mensaje de datos es confidencial, constituye un secreto industrial y/o profesional en terminos de la legislacion vigente y se encuentra dirigida exclusivamente al destinatario indicado en dicho mensaje. Si usted recibe esta informacion por error o si usted no es el destinatario del mensaje, favor de notificar al emisor, y destruyalo.
On Jul 9, 2020, at 1:32 PM, cdmeyer23 <notifications@github.com> wrote:
@Philliec459 <https://github.com/Philliec459>
Yes, I am working with the wolfcamp_single.py
If you look at the electrofacies source code here:
https://github.com/toddheitmann/PetroPy/blob/master/petropy/electrofacies.py <https://github.com/toddheitmann/PetroPy/blob/master/petropy/electrofacies.py>
it requires the import of numpy and pandas.
The example wolfcamp_single.py calls ptr.electrofacies on line 147:
logs = ptr.electrofacies(logs, f, electro_logs, 6,
log_scale = ['RESDEEP_N'])
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#7 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANSKYPDYVBR4EJFOQU4BXITR2YLMNANCNFSM4OVZ3LDA>.
cdmeyer23 commented
@Philliec459
Thank you so much helping me out, I think the only way to get around it for now is to use the setup that is provided. I didn't see that before. While it is frusterating to have to run it with older versions of numpy and pandas, which makes other code that is run with it not function properly, at least it works that way.
Thanks again!
Philliec459 commented
I think that it is easiest to load petropy to work from it.
As a Petrophysicist, this is probably the first python project that I worked on when I was getting started with Python. Now that I look back there are a lot of good things here that I want to pursue further.
I have my own GitHub with some petrophysical repositories that might help you too, but lately I am into Object Detection and so my repositories are deviating from petrophysics.
https://github.com/Philliec459?tab=repositories <https://github.com/Philliec459?tab=repositories>
My next goal will be to scan a log and do almost object detection picking out invervals, electrofacies, pay just from the log signatures. I will be using the Equinor Volve dataset for this.
Best Regards,
…_______________________________
E. Craig Phillips
CEO and Chief Petrophysicist
Crested Butte Petrophysical Consultants
459 Cisneros Lane
Crested Butte, CO 81224
USA
Office: +1 970-343-0730
Mobile: +1 970-343-0730
email: craig@cbpetro.com
Website: www.cbpetro.com
GitHub: https://github.com/Philliec459
The information contained in this electronic message is confidential, it constitutes a profesional and/or industrial secret in terms of the current legislation, and is intended for its recipient only. If you receive this message by mistake or if you are not the recipient thereof, please notify the sender and destroy it.
La informacion contenida en este mensaje de datos es confidencial, constituye un secreto industrial y/o profesional en terminos de la legislacion vigente y se encuentra dirigida exclusivamente al destinatario indicado en dicho mensaje. Si usted recibe esta informacion por error o si usted no es el destinatario del mensaje, favor de notificar al emisor, y destruyalo.
On Jul 10, 2020, at 10:43 AM, cdmeyer23 <notifications@github.com> wrote:
@Philliec459 <https://github.com/Philliec459>
Thank you so much helping me out, I think the only way to get around it for now is to use the setup that is provided. I didn't see that before. While it is frusterating to have to run it with older versions of numpy and pandas, which makes other code that is run with it not function properly, at least it works that way.
Thanks again!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#7 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANSKYPDBSQ6JF2OFGLIA2HDR25AJ7ANCNFSM4OVZ3LDA>.
dianaceroallard commented
@cdmeyer23 thank you for posting, I was stuck too in that electrofacies function, I am using Jupyter Notebook. Thanks to my friend Julio (st4tic0) who advised me to change "minibatch_input = components.as_matrix()" to "minibatch_input = components.to_numpy()" , it worked... at least the function electrofacies.py