umautobots/GTAVisionExport

Missing Documentation

dominikandreas opened this issue · 21 comments

dominikandreas commented

Hi, first of all thanks for sharing your work, I really appreciate it!

Would it be possible to add some documentation on how to compile the project and what the dependencies are?

cory8249 commented

+1 please help give some hint about how to use the code. Thanks a lot.

barcharcraz commented

Hi sorry for the lack of documentation. I'm writing up something more in depth but the short story is:

There are two components, the native and managed. The native component is probably the more useful one to you, since it deals with actually getting images and information from the game. The managed part mostly deals with uploading images to a NAS / S3 and the information to a postgres database (I can post a schema later today)

For the native portion:
You need Visual Studio 2015 Update 2 (VS 2017 preferred)
you need a recent version of cmake (I use 3.8)
you need Eigen
you need minhook
you need the scripthookv sdk

builds of minhook and scripthookv are provided in the deps folder, these are for VS2017 although they should work with 2015 as well.

Simply do the usual cmake build process and point cmake to Eigen (Eigen3_DIR I think)

The output will be a .asi file. You should put this beside the GTAV.exe file after making sure scripthookv is installed and working.

dominikandreas commented

Thanks! I already managed to compile the native part of the plugin using cmake 3.8 and Visual Studio 2015.

Note that for generating the visual studio solution using cmake, I had to use the following command:
cmake -DEIGEN3_INCLUDE_DIR=<eigen include dir> -G "Visual Studio 14 Win64" .
where <eigen include dir> is the path to the Eigen3 library (needs to be configured using cmake first).

The -G "Visual Studio 14 Win64" ensures that cmake generates a 64bit solution (32bit won't work afaik). The 14 is the version number for Visual Studio 2015 and needs to be changed if you want to use another version.

  • 👍2
barcharcraz commented

Yeah 32bit is pointless since GTA itself is 64bit.

The interface to the native library is in export.h and export.cpp. You can simply call them using the standard C FFI mechanism in whatever language you want to write in. We used C# and ScriptHookV.net but there's a lua binding as well. And ofc you can use C

caunion commented

@barcharcraz Hi, thanks for your work. I hope to ask if I can debug the program using Visual Studio's "attach to process"? Since every time we have to copy the generated .asi to the GTA 5's folder, and start the game all over again. Is there any short cut for debugging? Thanks.

barcharcraz commented
  • 👍1
caunion commented

Great. Thanks. I think hot reload could be a good way to debug.

barcharcraz commented

Also I've noticed that when gta is suspended or in a debugger all input to the computer seems to be delayed by a second or two. This is really annoying and I ended up working around it by using remote stub debugging.

muaz-git commented

Hello, first of all thank you for great project. So I am trying to build an asi file from native project.
I have
MS VS 2017
CMake 3.8.2
I downloaded Eigen from here.

So first I tried to build Eigen,
I made a 'build_dir' in 'source_dir' (where source_dir is the extraction folder), then I ran cd build_dir and cmake .. I get these logs(apparently seems to be successful to me). But then when I tried to run make install, I get this message and I was not able to properly build Eigen.

So then I tried to build native project as @dominikandreas suggested.
I made a 'build_dir' in 'source_dir' (where source_dir is the extraction folder), then I ran cd build_dir and cmake -DEIGEN3_INCLUDE_DIR=/cygdrive/c/Users/Admin/Downloads/eigen -G "Visual Studio 15 Win64" .. I get these logs (seems like I have some problem in linking to Eigen).

Sorry I am really new to building projects, I would really appreciate if any of you please help me out here. Thank you.

dominikandreas commented

It seems you're using cygwin, which is not necessary (since we're compiling with visual studio and not gnu/linux toolchains). Get cmake for windows from here: https://cmake.org/download/, use that from a regular windows cmd and use native windows paths (not /cygdrive/c/..) for the parameters.

barcharcraz commented
muaz-git commented

Thank you for your quick response. So I used following:

  1. ran the command cmake -DEIGEN3_INCLUDE_DIR=<path to Eigen extraction folder> -G "Visual Studio 15 Win64" .. in build_dir and these logs were generated I get this content in the build_dir.
  2. Then I tried to run nmake in build_dir as well as in native folder but it gives me error that U1064 Makefile not found.
    Also I am not abe to see any asi file. I would appreciate the help.
dominikandreas commented

You need to open the .sln with visual studio and build the solution. That should generate the .asi plugin

  • 👍1
IanKirwan commented

GTAVisionExport (native) build steps

Note: there may be some unessesary steps that could be removed from this procedure at some point, but this is what I did. Also I have a tendency write Linux style paths. All paths (except URLs) should be Windows style.

My set-up:

GTX770, Windows 10 64bit, Visual Studio 2017 (community edition), cmake 3.9.0, GTAV version 1.0.231.0

Needed tools and libraries

AsiLoader and ScriptHookV : http://www.dev-c.com/files/ScriptHookV_1.0.1103.2.zip
cmake : https://cmake.org/download/
Eigen3 : http://bitbucket.org/eigen/eigen/get/3.3.1.tar.bz2

Build steps

  1. git clone https://github.com/umautobots/GTAVisionExport (latest! as of 4 Aug 2017)
  2. Extract Eigen3 somwhere convenient (Your GTAVisionExport folder is as good as any).
  3. Extract ScriptHookV archive and drop the files in 'bin' into your GTAV exe folder.
  4. Run cmake (cmake-gui) from your Windows start menu.
  5. Hit 'Browse Source' and select your GTAVisionExport/native folder.
  6. Hit 'configure' (first time around it will fail but dont worry).
  7. Choose project generator 'Visual Studio 15 2017 Win64' and keep the option 'use default native compilers'
  8. After the fail dialog, modify the EIGEN3_INCLUDE_DIR to point to your Eigen3 folder.
  9. Run 'configure' followed by 'generate'.
  10. cmake should now have generated the Visual Studio solution into GTAVisionExport/build.
  11. Open 'GTANativePlugin.sln' in Visual Studio.
  12. Select 'release' from the 'Solution Configurations' drop down.
  13. Edit GTAVisionNative project properties/configuration properties/c/c++/additional include dirs in VS to add the GTAVisionExport/src folder (this allows VS to find MinHook.h)
  14. Edit GTAVisionNative project properties/configuration properties/linker/input/additional dependencies to add :
    "..\..\deps\libMinHook.x64.lib"
  15. Press F6 to build the solution. it should now succeed and the products should be in 'GTAVisionExport\native\build\src\Release'
  16. Copy GTAVisionNative.asi & GTAVisionNative.lib to your GTAV exe folder.
  17. Run GTAV.
  18. Get to a place where you want to grab frames and press 'l' (lowercase 'L') to grab a frame. GTAVisionExport should now create color.raw, stencil.raw and depth.raw files in your GTAV exe folder.

HTH

tyzaizl commented

Hello, thank you for sharing this work. I got the color.raw, depth.raw and stencil.raw files, but how should I decode the files to normal images or how to use them?

  • 👍2
IanKirwan commented

@tyzaizl
You're supposed to be able to open them in Photoshop, but there is no dimensional info in at least the depth file that PS can use, so you end up putting something in and the depth map comes out garbled. It would be good if @barcharcraz could tell us how they are decoding this info.

  • 👍2
tyzaizl commented

got it, and Is there any matlab or python script to extract the image from the raw data.
In matlab, I try to read the raw data but got meanless images,
here the code:
raw_data_path = fullfile(gta5_root, 'color.raw'); fid=fopen(raw_data_path, 'r'); img=fread(fid,[1080 1920]);
Is there any advices?

dominikandreas commented

You should be able to read the file in python and parse it as a numpy array using arr=np.fromstring(str, dtype=...). You'll have to adjust the dtype depending on the buffer. Once you have that, you can resize the array and display it using matplotlibs pyplot, i.e. plt.imshow(arr)

tyzaizl commented

@dominikandreas
Thanks for replay.
What do you mean "dtype depending on the buffer"?
For color.raw, what dtype should be, float or double?

dominikandreas commented

If I remember correctly, color was uint8, depth was float32, not sure about stencil. You can figure it out which dtype you need for which buffer by just trying it out and visualizing the result.

racinmat commented

stencil is also uint8, I think