File sizes
pioro opened this issue · 4 comments
Hi Urh,
I'm doing some test of your code as I want to include it in my DevOps Oracle presentation.
I know from your comments that git doesn't work for now due to size of files set to 0.
I compiled a latest commit and this is what I can see
ls -l on file show 0 as a size
[oracle@linuxsource test121]$ ls -l /home/oracle/test121/plsql/PIORO/PROCEDURE/TEST.SQL
-rw-r--r--. 1 oracle oinstall 0 Jan 10 06:58 /home/oracle/test121/plsql/PIORO/PROCEDURE/TEST.SQL
ls -l on directory above show actual size 88
[oracle@linuxsource test121]$ ls -l /home/oracle/test121/plsql/PIORO/PROCEDURE/
total 0
-rw-r--r--. 1 oracle oinstall 88 Jan 10 06:58 TEST.SQL
Are you working on code to make it consistent now ?
If you need help, let me know and I will take a look on it. I didn't code in C for while
but maybe I will be able to help you.
Great Job !!!!
regards,
Marcin
I found a bit strange behaviour, I restarted it with -f and LOGLEVEL=DEBUG
now is working fine ;)
[oracle@linuxsource ~]$ ls -l /home/oracle/test121/plsql/PIORO/PROCEDURE/
total 0
-rw-r--r--. 1 oracle oinstall 88 Jan 10 06:58 TEST.SQL
Hi,
The behaviour described is expected, although it may seem strange at first. It depends on filesize mount option.
All uncached files will report filesize of n bytes, where n is the value of filesize mount option.
Files are put into cache whenever they are opened. Except if you specify filesize=-1 - in this case files are put into cache not only on open() but also whenever they are listed (using ls for instance).
"cache" that I'm describing is actually just a regular folder on physical filesystem (such as ext4), specified by temppath mount option. You can issue ls in that cached folder to see cache contents.
In release 1.0, the cached file was deleted on close(). This is changed if you compile from latest source so that cache is deleted on umount. Features mentioned from this point on are available in latest source, but not in release 1.0:
That means, if you specify filesize >= 0, it will report:
- correct file size for all files that were opened since mount.
- incorrent (value of
filesize) for files that were never opened since mount.
Contents of temppath can be reused after umount. Actualy, specifying (currently undocumented) keepcache mountoption will not delete contents of cache directory.
This is important, becase, as mentioned, git only works if filesystem provides (strictly) correct filesizes.
So, specifying filesize=-1 will make git work as well =)
This, however, has terrible effect on peformance if the cache is empty. But, once every .sql file is cached it is as fast as specifying filesize=0. This basically means that if you use crontab script which does git commit once a day or so, it will be ok (except for the first commit). In such scenario I would recommend setting filesize=-1,keepcache.
In the other case, if you wish to work with .SQL files interactively (like using VIM etc), I wold recommend to use filesize=0 (without keepcache).
When is the cached version invalidated?
For every cached .SQL file ddlfs also stores last_ddl_time of that object as it was when it was cached. On every file open and file listing, the last_ddl_time from dba_objects is compared to that written in cache. If it is not the same, cached version is updated (source is downloaded from database).
I would be happy to look into the issue if you find behaviour inconsistent with what I just wrote. Also, any ideas for improvement are appreciated - especially if they come in the form of code/patches :)
p.s. I'm a bit busy currently, but i do intend to put more effort in this project once I lower my backlog in "commercial" work :)
I'm still not sure what would be a better default for next release, filesize=-1,keepcache to make most software work or filesize=0,nokeepcache which is cleaner, faster and makes a lot of software (but not git) work :)
this was a bug - filesizes were always reported correctly, even when filesize parameter was not set to -1.
this was fixed in today's commit.