Bug Report: treeStructure.py does not match post build

Question

Bug Report: treeStructure.py does not match post build

Closed this issue 6 years ago · 11 comments

Brief summary of issue

Noticed that after building a rpm and installing the package into my env the file treeStructure.py does not match the copy I had in vfatqc-python-scripts that was used to build the package.

Types of issue

Bug report (report an issue with the code)
Feature request (request for change which adds functionality)

Expected Behavior

If I edit a file and build the package, it should reflect the changes I made.

Current Behavior

Editing treeStructure.py and then building does not update the file in the build area or in the installed venv; see example below.

Steps to Reproduce (for bugs)

add some line to treeStructure.py
make && make rpm 2>&1 | tee buildLog.log
pip install $BUILD_HOME/vfatqc-python-scripts/rpm/gempython_vfatqc-2.0.0.tar.gz

I noticed in two locations the treeStructure.py file doens't match, e.g.:

In the installed venv:

% diff $BUILD_HOME/vfatqc-python-scripts/treeStructure.py $VIRTUAL_ENV/lib/python2.7/site-packages/gempython/vfatqc/treeStructure.py
19,22c19
< 
<         self.calSF = array( 'i', [0] )
<         self.gemTree.Branch( 'calSF', self.calSF, 'calSF/I')
< 
---
>         
25,31c22
< 
<         self.isCurrentPulse = array( 'i', [ 0 ] )
<         self.gemTree.Branch( 'isCurrentPulse', self.isCurrentPulse, 'isCurrentPulse/I')
<        
<         self.isZCC = array( 'i', [0] )
<         self.gemTree.Branch( 'isZCC', self.isZCC, 'isZCC/I' )
< 
---
>         
59,62c50
< 
<         self.trimPolarity = array( 'i', [ 0 ] )
<         self.gemTree.Branch( 'trimPolarity', self.trimPolarity, 'trimPolarity/I' )
< 
---
>         
89,91d76
<     
<         self.vthr = array( 'i', [ 0 ] )
<         self.gemData.Branch( 'vthr', self.vthr, 'vthr/I' )
113,114d97
<         if "calSF" in kwargs:
<             self.calSF[0] = kwargs["calSF"]
117,120d99
<         if "isCurrentPulse" in kwargs:
<             self.isCurrentPulse[0] = kwargs["isCurrentPulse"]
<         if "isZCC" in kwargs:
<             self.isZCC[0] = kwargs["isZCC"]
126c105
<             self.link[0] = kwargs["link"]
---
>             self.link[0] = kwargs["kwargs"]
137,138d115
<         if "trimPolarity" in kwargs:
<             self.trimPolarity[0] = kwargs["trimPolarity"]
157,158d133
<         if "vthr" in kwargs:
<             self.vthr[0] = kwargs["vthr"]

And then in the build area:

% diff $BUILD_HOME/vfatqc-python-scripts/treeStructure.py $BUILD_HOME/vfatqc-python-scripts/rpm/gempython/vfatqc/treeStructure.py 
19,22c19
< 
<         self.calSF = array( 'i', [0] )
<         self.gemTree.Branch( 'calSF', self.calSF, 'calSF/I')
< 
---
>         
25,31c22
< 
<         self.isCurrentPulse = array( 'i', [ 0 ] )
<         self.gemTree.Branch( 'isCurrentPulse', self.isCurrentPulse, 'isCurrentPulse/I')
<        
<         self.isZCC = array( 'i', [0] )
<         self.gemTree.Branch( 'isZCC', self.isZCC, 'isZCC/I' )
< 
---
>         
59,62c50
< 
<         self.trimPolarity = array( 'i', [ 0 ] )
<         self.gemTree.Branch( 'trimPolarity', self.trimPolarity, 'trimPolarity/I' )
< 
---
>         
89,91d76
<     
<         self.vthr = array( 'i', [ 0 ] )
<         self.gemData.Branch( 'vthr', self.vthr, 'vthr/I' )
113,114d97
<         if "calSF" in kwargs:
<             self.calSF[0] = kwargs["calSF"]
117,120d99
<         if "isCurrentPulse" in kwargs:
<             self.isCurrentPulse[0] = kwargs["isCurrentPulse"]
<         if "isZCC" in kwargs:
<             self.isZCC[0] = kwargs["isZCC"]
126c105
<             self.link[0] = kwargs["link"]
---
>             self.link[0] = kwargs["kwargs"]
137,138d115
<         if "trimPolarity" in kwargs:
<             self.trimPolarity[0] = kwargs["trimPolarity"]
157,158d133
<         if "vthr" in kwargs:
<             self.vthr[0] = kwargs["vthr"]

Context (for feature requests)

Doesn't properly incorporate changes.

Your Environment

Version used: rpc-playground c1d5153
Shell used: /bin/zsh

Answer 1 · 2018-06-08T09:41:45.000Z

Could you please specify exact pip command you used and exact output you got? It looks like you don't change the version of the package so pip might just ignore it as it has installed this package with this version already

Answer 2 · 2018-06-08T09:44:29.000Z

That would potentially explain why the first diff exists. But the second diff shown is the vfatqc-python-scripts sub-directory. e.g. it's not in my venv it's under:

$BUILD_HOME/vfatqc-python-scripts/rpm/gempython/vfatqc/treeStructure.py

So the issue is independent of me doing a pip install into my venv.

Answer 3 · 2018-06-08T10:00:26.000Z

hmmm this is interesting. What if you run make cleanrpm && make rpm?
Maybe some intermediate precompiled files do not receive an update (e.g. those *.pyc)

Answer 4 · 2018-06-08T10:33:28.000Z

I’ll give it a go in the afternoon

Answer 5 · 2018-06-08T12:02:12.000Z

[dorney@gem904qc8daq]~/scratch0/CMS_GEM/CMS_GEM_DAQ/vfatqc-python-scripts% make cleanrpm && make rpm                                                                                                       
[dorney@gem904qc8daq]~/scratch0/CMS_GEM/CMS_GEM_DAQ/vfatqc-python-scripts% diff $BUILD_HOME/vfatqc-python-scripts/treeStructure.py $BUILD_HOME/vfatqc-python-scripts/rpm/gempython/vfatqc/treeStructure.py 
19,22c19
< 
<         self.calSF = array( 'i', [0] )
<         self.gemTree.Branch( 'calSF', self.calSF, 'calSF/I')
< 
---
>         
25,31c22
< 
<         self.isCurrentPulse = array( 'i', [ 0 ] )
<         self.gemTree.Branch( 'isCurrentPulse', self.isCurrentPulse, 'isCurrentPulse/I')
<        
<         self.isZCC = array( 'i', [0] )
<         self.gemTree.Branch( 'isZCC', self.isZCC, 'isZCC/I' )
< 
---
>         
59,62c50
< 
<         self.trimPolarity = array( 'i', [ 0 ] )
<         self.gemTree.Branch( 'trimPolarity', self.trimPolarity, 'trimPolarity/I' )
< 
---
>         
89,91d76
<     
<         self.vthr = array( 'i', [ 0 ] )
<         self.gemTree.Branch( 'vthr', self.vthr, 'vthr/I' )
113,114d97
<         if "calSF" in kwargs:
<             self.calSF[0] = kwargs["calSF"]
117,120d99
<         if "isCurrentPulse" in kwargs:
<             self.isCurrentPulse[0] = kwargs["isCurrentPulse"]
<         if "isZCC" in kwargs:
<             self.isZCC[0] = kwargs["isZCC"]
126c105
<             self.link[0] = kwargs["link"]
---
>             self.link[0] = kwargs["kwargs"]
137,138d115
<         if "trimPolarity" in kwargs:
<             self.trimPolarity[0] = kwargs["trimPolarity"]
157,158d133
<         if "vthr" in kwargs:
<             self.vthr[0] = kwargs["vthr"]

The issue persists when attempting a make cleanrpm

Answer 6 · 2018-06-08T12:36:34.000Z

You're not actually reinstalling the package (because the package version hasn't changed)
So you can bump the version number (but don't commit the change!), before making the RPM, or you force reinstall (pip install -I), which would be my suggestion for developing

Answer 7 · 2018-06-08T12:53:27.000Z

You're not actually reinstalling the package (because the package version hasn't changed)
So you can bump the version number (but don't commit the change!), before making the RPM, or you force reinstall (pip install -I), which would be my suggestion for developing

I don't follow. The issue occurs before I would get to the pip install -I stage. As reported above the difference is in:

$BUILD_HOME/vfatqc-python-scripts/rpm/gempython/vfatqc/treeStructure.py

Which is not in my venv.

Do you mean that for the built rpm to properly reflect any local edits I need to change the version number and then execute:

make cleanrpm && make rpm

If so where is the file I would need to change the version number in and can instructions here be updated to reflect this?

Answer 8 · 2018-06-08T12:59:28.000Z

Hmm, can't reproduce, here's what I tried:

make && make rpm with current vanilla:

[sturdy@gem904daq02 sw/vfatqc-python-scripts]% diff treeStructure.py ./rpm/gempython/vfatqc/treeStructure.py
[sturdy@gem904daq02 sw/vfatqc-python-scripts]%

Now i introduce some change:

[sturdy@gem904daq02 sw/vfatqc-python-scripts]% git diff
diff --git a/treeStructure.py b/treeStructure.py
index be81740..d65aed4 100644
--- a/treeStructure.py
+++ b/treeStructure.py
@@ -17,6 +17,15 @@ class gemTreeStructure:
         self.calPhase = array( 'i', [ 0 ] )
         self.gemTree.Branch( 'calPhase', self.calPhase, 'calPhase/I' )
         
+        self.calPhase2 = array( 'i', [ 0 ] )
+        self.gemTree.Branch( 'calPhase2', self.calPhase2, 'calPhase2/I' )
+        
+        self.calPhase3 = array( 'i', [ 0 ] )
+        self.gemTree.Branch( 'calPhase3', self.calPhase3, 'calPhase3/I' )
+        
+        self.calPhase4 = array( 'i', [ 0 ] )
+        self.gemTree.Branch( 'calPhase4', self.calPhase4, 'calPhase4/I' )
+        
         self.Dly = array( 'i', [ -1 ] )
         self.gemTree.Branch( 'Dly', self.Dly, 'Dly/I' )

The version in the rpm directory is also showing the differences:

[sturdy@gem904daq02 sw/vfatqc-python-scripts]% diff treeStructure.py ./rpm/gempython/vfatqc/treeStructure.py
20,28d19
<         self.calPhase2 = array( 'i', [ 0 ] )
<         self.gemTree.Branch( 'calPhase2', self.calPhase2, 'calPhase2/I' )
<         
<         self.calPhase3 = array( 'i', [ 0 ] )
<         self.gemTree.Branch( 'calPhase3', self.calPhase3, 'calPhase3/I' )
<         
<         self.calPhase4 = array( 'i', [ 0 ] )
<         self.gemTree.Branch( 'calPhase4', self.calPhase4, 'calPhase4/I' )
<

Next I again do make rpm

[sturdy@gem904daq02 sw/vfatqc-python-scripts]% diff treeStructure.py ./rpm/gempython/vfatqc/treeStructure.py
[sturdy@gem904daq02 sw/vfatqc-python-scripts]%

Answer 9 · 2018-06-08T13:03:16.000Z

Initially after reading your most recent comment I was thinking it was because of make not being run (make populates pkg which is then copied into the rpm build area), but as seen above, I only re-ran make rpm and things "worked"

Answer 10 · 2018-06-08T13:09:13.000Z

Initially after reading your most recent comment I was thinking it was because of make not being run (make populates pkg which is then copied into the rpm build area), but as seen above, I only re-ran make rpm and things "worked"

Which branch are you trying this on?

Okay it wasn't quiet clear, just to check my own understanding the procedure should be:

execute make && make rpm,
introduce some change,
execute make rpm
profit...?

Answer 11 · 2018-06-08T13:13:51.000Z

Current pointer is:

* gemdaq-develop              0aa28de [gemdaq/develop] bump gembuild submodule

make rpm should pull in make default (make) if nothing else has been done (if not, then this will need to be fixed), so all you should have to do would be make rpm
~~Actually, the make pip target hasn't been updated for the package, so this could be what's happening (provided you're not seeing this problem with make rpm but are with make pip)~~, nope, pip target still has preprpm as a dependency, so this isn't the issue...