Maximum path length limited to 255 characters

We are running into a weird error on Linux machines when calling compile /full/path/to/file.dss, when the path is more than 255 characters. The error returned by OpenDSS is not obvious, since it just says it cannot open /full/path/to/file.dss (without truncating).

I'm assuming this is not a straightforward fix, since OpenDSS is just calling built-in pascal functions:

dss_capi/Version7/Source/Executive/ExecHelper.pas

Lines 347 to 406 in cf442f6

    
           FUNCTION DoRedirect(IsCompile:Boolean):Integer; 
        
           //  This routine should be recursive 
        
           //  So you can redirect input an arbitrary number of times 
        
           // If Compile, makes directory of the file the new home directory 
        
           // If not Compile (is simple redirect), return to where we started 
        
           VAR 
        
               Fin:TextFile; 
        
               ParamName,  InputLine, CurrDir, SaveDir : String; 
        
               LocalCompFileName  : String; 
        
               InBlockComment : Boolean; 
        
           Begin 
        
               Result := 0; 
        
               InBlockComment := FALSE;  // Discareded off stack upon return 
        
               // Therefore extent of block comment does not extend beyond a file 
        
               // Going back up the redirect stack 
        
               // Get next parm and try to interpret as a file name 
        
               ParamName := Parser.NextParam; 
        
               ReDirFile := ExpandFileName(Parser.StrValue); 
        
               IF ReDirFile <> '' THEN 
        
               Begin 
        
                 SaveDir :=  GetCurrentDir; 
        
                 TRY 
        
                     AssignFile(Fin, ReDirFile); 
        
                     Reset(Fin); 
        
                     If IsCompile Then Begin 
        
                        LastFileCompiled := ReDirFile; 
        
                        LocalCompFileName:= ReDirFile; 
        
                     End; 
        
                 EXCEPT 
        
                    // Couldn't find file  Try appending a '.dss' to the file name 
        
                    // If it doesn't already have an extension 
        
                    IF   Pos('.', ReDirFile)=0 
        
                    THEN Begin 
        
                       ReDirFile := ReDirFile + '.dss'; 
        
                       TRY 
        
                           AssignFile(Fin, ReDirFile); 
        
                           Reset(Fin); 
        
                       EXCEPT 
        
                           DoSimpleMsg('Redirect File: "' + ReDirFile + '" Not Found.', 242); 
        
                           SolutionAbort := TRUE; 
        
                           Exit; 
        
                       End; 
        
                    End 
        
                    ELSE Begin 
        
                          DoSimpleMsg('Redirect File: "'+ReDirFile+'" Not Found.', 243); 
        
                          SolutionAbort := True; 
        
                          Exit;  // Already had an extension, so just Bail 
        
                    End; 
        
                 END;

Weird, FPC's MAX_PATH is 4096 on Linux and still fails. FileExists(ReDirFile) returns true though. I'll try to look for an alternative to AssignFile.

What I found so far is that it doesn't seem to be related to string encoding or anything like it. Using a TFileStream seems to work, I only worry about the encoding of the contents.

Related but closed for 3.0.0 (we're on 3.0.4): https://bugs.freepascal.org/view.php?id=24885

FPC 3.2 was expected this year. If it's indeed a bug, might be worth report it before the release (assuming it still existing in the development version).

I'm considering using a stream as a fallback if FileExists returns true -- just load it in a TStringList and iterate over the lines.

It will use a bit more memory since it loads all lines at once. I assume that's not an issue.

We might be able to get away with just removing this one line:

dss_capi/Version7/Source/Executive/ExecHelper.pas

Line 369 in cf442f6

ReDirFile := ExpandFileName(Parser.StrValue);

I can't think of a reason this function would need to call ExpandFileName. Even if it does, we can add it as a separate try except and we should be good.

Yeah, ExpandFileName doesn't look it does anything useful here, weird. There might be some corner case.

Searching further, this thread from 2018 mentions exactly the same error (the issue is with the Reset function): https://forum.lazarus.freepascal.org/index.php?topic=39522.0
I didn't find the exact bug report on https://bugs.freepascal.org/ but the ones I found suggest using TFileStream too, Assign and Reset are deprecated and won't be fixed (!).

Besides the change to DoRedirect, I ported some other changes to 0.10.x and created a branch (master is 0.11 dev), if you want to give it a try: 96bc0bb

It tries the following:

given filename
expanded filename
if the expanded filename does exist, fallback to a string list -- won't affect other files
otherwise, fail

I tested with just a file name and a full long path:

'compile 40242748-21fd-4a61-8ad9-cefdd3ff6a05-40242748-21fd-4a61-8ad9-cefdd3ff6a05-40242748-21fd-4a61-8ad9-cefdd3ff6a05-40242748-21fd-4a61-8ad9-cefdd3ff6a05-40242748-21fd-4a61-8ad9-cefdd3ff6a05-40242748-21fd-4a61-8ad9-cefdd3ff6a05.dss'
compile /home/meira/40242748-21fd-4a61-8ad9-cefdd3ff6a05-40242748-21fd-4a61-8ad9-cefdd3ff6a05-40242748-21fd-4a61-8ad9-cefdd3ff6a05-40242748-21fd-4a61-8ad9-cefdd3ff6a05-40242748-21fd-4a61-8ad9-cefdd3ff6a05-40242748-21fd-4a61-8ad9-cefdd3ff6a05/40242748-21fd-4a61-8ad9-cefdd3ff6a05-40242748-21fd-4a61-8ad9-cefdd3ff6a05-40242748-21fd-4a61-8ad9-cefdd3ff6a05-40242748-21fd-4a61-8ad9-cefdd3ff6a05-40242748-21fd-4a61-8ad9-cefdd3ff6a05-40242748-21fd-4a61-8ad9-cefdd3ff6a05.dss'

I'll add a couple of minor fixes from upstream tomorrow on that branch, and hopefully merge a monstrous changeset I have locally too on master.

Since OpenDSS uses SetCurrentDir, DoRedirect is probably broken in some multi-threading scenarios. I'll open a ticket about it.

You are awesome @PMeira! I'll try the fix.

@kdheepak I tagged it so the binaries should be here soon https://github.com/dss-extensions/dss_capi/releases

	FUNCTION DoRedirect(IsCompile:Boolean):Integer;

	// This routine should be recursive
	// So you can redirect input an arbitrary number of times

	// If Compile, makes directory of the file the new home directory
	// If not Compile (is simple redirect), return to where we started

	VAR
	Fin:TextFile;
	ParamName, InputLine, CurrDir, SaveDir : String;
	LocalCompFileName : String;
	InBlockComment : Boolean;

	Begin
	Result := 0;
	InBlockComment := FALSE; // Discareded off stack upon return
	// Therefore extent of block comment does not extend beyond a file
	// Going back up the redirect stack

	// Get next parm and try to interpret as a file name
	ParamName := Parser.NextParam;
	ReDirFile := ExpandFileName(Parser.StrValue);

	IF ReDirFile <> '' THEN
	Begin

	SaveDir := GetCurrentDir;

	TRY
	AssignFile(Fin, ReDirFile);
	Reset(Fin);
	If IsCompile Then Begin
	LastFileCompiled := ReDirFile;
	LocalCompFileName:= ReDirFile;
	End;
	EXCEPT

	// Couldn't find file Try appending a '.dss' to the file name
	// If it doesn't already have an extension

	IF Pos('.', ReDirFile)=0
	THEN Begin
	ReDirFile := ReDirFile + '.dss';
	TRY
	AssignFile(Fin, ReDirFile);
	Reset(Fin);
	EXCEPT
	DoSimpleMsg('Redirect File: "' + ReDirFile + '" Not Found.', 242);
	SolutionAbort := TRUE;
	Exit;
	End;
	End
	ELSE Begin
	DoSimpleMsg('Redirect File: "'+ReDirFile+'" Not Found.', 243);
	SolutionAbort := True;
	Exit; // Already had an extension, so just Bail
	End;

	END;