deislabs/spiderlightning

Implement a better install routine to handle required runtime dependencies of windows in install.ps1 for slight.exe

D1esel-Dev opened this issue · 1 comments

D1esel-Dev commented

Describe the solution you'd like

Currently for a Windows 'execute' of the slight.exe, (which is 'installed' via a ps1 script to handle adding it to PATH and used in CMD) requires other dependencies to run shown here #378. Most systems with/without build tools (usually) come with a vendor copy of OpenSSL (primarily Linux, MSYS or git bash/gui) to link with your tools and run your developed bins. However, most systems are NOT expecting you to need most these tools Out-The-Box and therefore excluded.

It would be useful to have the 'install.ps1' script automate the help in discovery and resolution for slights' OpenSSL dependency. Thus making it way less a head-ache to manually have to check for or install OpenSSL, because lets face it, that process is tedious enough to cause multiple breaks along the way and most of us are just too lazy...

Therefore, breaks in slight.exe can simply occur due to:

  • OpenSSL simply not existing in the same run-time environment of slight.exe
  • OpenSSL was installed under a different user environment, i.e. an admin account
  • Runtime Environment of slight.exe doesn't know where to look for OpenSSL (expected PATH variable to exist, if OpenSSL exists, but not always a case)
  • The run-time environment is ok and OpenSSL found, but the version required and built in slight.exe, mismatches the minimal required found version of the run-time OpenSSL (usually only an issue when the end user needs to keep specific versions for internal reasons)

NOTE That this 'install.ps1' does produce expected behavior by expecting OpenSSL to be called dynamically, ergo it assumes OpenSSL is installed, in the environment user path and ready to be invoked. A modification of the 'install.ps1' to not assume this, is the goal. Most end users(YOU), use or have their versions and compiled builds of OpenSSL made specifically for their runtime environment. Thus to accommodate for any custom setup that needs to run for a bin in their environment, like using remote bin(s) for development. Hence, a custom OpenSSL build and environment used as a dep for bin x specifically, will not be configured to work for bin y, unless configured to do so.

The manual solution without implementing into install.ps1 (may still break, may get things working):

  1. End user installs a pre-built vendor OpenSSL (at least >= version:1.1.1) in the same run-time environment of slight.exe
  2. The environment on install should be configured automatically to add its' install location to PATH, else have to manually add it to the current user environment PATH
  3. Run slight.exe
  4. profit!?

A way less hair-pulling method can and should exist.

Trust me, no one solution is concrete nor friendly enough (at least to me) to accomplish this correctly, but we can certainly allow for any break handling of anomalies by modifying the 'install.ps1' script. I know I don't want to be dealing with the manual solution every time.

Additional context

My Expectations:

  1. From my understanding: Execution of 'slight.exe' will look for an OpenSSL candidate of at least version 1.1.1 by invoking a 'openssl' cmd needed for run-time. The system then searches for this 'openssl' exe file name by searching in the URLs of the set current user environments' PATH variables. Once matched, 'openssl' can execute as a dep within slight.exe, and we should be getting I/O.
  2. Slight.exe breaks with no I/O when:
    • No OpenSSL candidate at all
    • An OpenSSL candidate is of lower version than what is required to execute slight.exe
    • An OpenSSL candidates' path is not exposed to the environment to find a match to 'openssl'
    • An OpenSSL candidate is exposed as user x's environment on the system, but not exposed as the current user y's environment run-time for slight.exe
  3. The install.ps1 script breaks when:
    • Update-SessionEnvironment is invoked because it assumes the user has chocolatey as a package manager in their windows environment, and should be removed

The Problem:

Windows (IMO) is the worst on making sure runtime deps are available when a bin makes a call to a linked dep , because most redistribution runtimes that contain development build tools such as OpenSSL are; not properly installed, exist by default or not exposed to PATH.

Following the simple (<- hahaha) ways to obtain OpenSSL here, specifically explained as how to acquire OpenSSL to use for building rust-openssl as a needed dependency.

I would like to see each windows method added as part of the install.ps1 script for the slight.exe binary as options to set up before the slight install finishes. At the minimum detect and warn about inconsistency in dependency existence, and at most provide the full resolution process of acquiring OpenSSL when it cannot be found or not in current user environment. This will make an easier way to inform users of breaks and manage optimal solutions for missing dependencies before slight install finishes.

I personally will assign myself to get started in testing the methods for windows platform only ATM, and how to implement them into 'install.ps1'.

How I will set, use and test an Environment:

  • Windows 10 IoT Enterprise LTSC, Build: 21H2
  • User local account
  • Clean install:
    • execute with no expectations and changes to environment
    • validate result (did it 'run'?)
    • reflect on the 'why' of the result to form a solution
    • perform a solution change to the environment (installing dep, changing PATH variables)
    • record what's changed in process, how and weight the solutions feasibility
    • validate result (did it 'run'?)
    • if produced desired execution results and changes:
      • mimic the method's integration into 'install.ps1'
      • validate the process added to the script^
    • rinse and repeat for each method till a clean, ache proof process for a slight install on windows caring for dependencies is created
  • My Q's and C's:
    • Will being part of any different user group affect the results and changes?
    • Will install of OpenSSL require an elevated user or can you install as current user?
    • I will try to make it for local user only based changes
    • Does an install in local user x environment under an elevated user y, set the PATH environment for the elevated user y only?

I can post my findings here, or may I recommend to create a project for workflow.

Mock-up of proposed 'install.ps1' script additions for validations and installs of OpenSSL:

  1. add a check to find any candidates of OpenSSL in default locations, Usually in root C, ~/Program Files or ~/Program FIles(x86), and named as /{platform}OpenSSL-{version} where platform for this case is WinXX xx=architecture and version is the version number. (a version is not always a part of the name but can acquired from with in the path through a file or execute of 'opensssl -v' )
  2. if exists:
    • show user the system found candidate(s) location(s) and their version(s)
    • add a validation to filter out any incompatible versions of found candidate(s)
    • add a validation checking if candidate(s) location is already exposed to the current user environment as a PATH variable
      • notify user of candidate(s) which are not in current user PATH environment : imply this as the reason I/O of slight.exe wont work and it needs to be added
    • if both, compatible candidate exists and its' location exposed in current user environment PATH : notify user that no configurations needs done
      • continue to step 6
    • if only one candidate is found, set its' location to be used : notify user of the set location
      • continue to step 4
    • if multiple candidates, allow user to select a candidate wanted : user input selection of a candidate, selected candidates' location will be used : notify user of the set location
      • continue to step 4
  3. if NOT exists: notify user no candidate(s) found
    • prompt the user if there is a custom location of OpenSSL that exists, remote or locally somewhere or if a new install wanted: user input if to use custom candidate
    • if yes:
      • warn about linking to custom candidate(s) : notify this user, If custom candidate is ever moved or modified in some way, the environment wont find it even if its' PATH is set, you will need the new location re-set for the PATH
      • ask user to provide location of the custom candidate wanted : user input or spawn directory explorer?
      • validate if the custom candidates' location actually exists : notify user on validate
      • if so, its location will be used : notify user the location that will be used
      • continue to step 4
    • if no:
      • provide the option to install a pre-compiled binary from a method mentioned above : user input selection to install or not install
      • check the current systems' architecture before making a URLfor the correct candidate download (should be done before searching for candidates )
      • installing the candidate: user input choose 'yes'
        • install a candidate via chosen method: (todo for other methods)
          • Windows MSVC method:
            • download a MSI installer for OpenSSL from: base-url=https://slproweb.com/download/ , content: WinXXOpenSSL-V_V_V.msi, match the XX with architecture of system and V is the version numbers
            • validate the downloaded OpenSSL installers' md5sums - A wise bird once told me to NEVER skip integrity checks : notify user on validate
            • if mis-match in md5sums: notify user of mismatch and need for a re-download
              • re-download
              • validate again
              • repeat for a max amount of tries until validated
                • continue on validation
            • install the MSI package via Start-Process : notify on success
            • the MSI installer should set its' install location to the current user PATH environment, but we need to check anyways
            • validate if the installer install added its' install location to the current user environment PATH : notify user if install already configured correctly for your environment
              • continue to step 6 on validation above
            • else, the newly installed location will be used : notify user of the set location
              • continue to step 4
          • Windows vcpkg method:
            • todo
          • Windows GNU (MinGW) method:
            • todo
      • don't install (a user is not forced to install a candidate this way): user input choose 'no'
        • notify the user that a manual installation of OpenSSL and setting of environment PATH, will still need to be done in order to fully execute slight.exe
        • continue to step 6
  4. set the passed saved location from steps 2-3 to the current user environment PATH variable, it is either: user selected, default/found or newly installed
  5. validate the Path location of a candidate was set correctly ^ : notify user on validation
  6. continue with the slight install
  7. profit!
Mossaka commented

Wow, @D1esel-Dev , thank you for your detailed write-up!!

I will forward this to @danbugs who worked on the installation script for Windows to take a look.

  • 👍1