[Windows Support]module 'os' has no attribute 'getpgid'

[eerkaijun]

When running env.reset(), I encounter this error:

Initializing new Carla server...                                                                                        
FATAL ERROR while launching server: <class 'AttributeError'>                                                            
Error during reset: Traceback (most recent call last):                                                                    
File "C:\Eer Kai Jun\Autonomous Driving\env\lib\site-packages\macad_gym\carla\multi_env.py", line 579, in reset         self._init_server()                                                                                                   
File "C:\Eer Kai Jun\Autonomous Driving\env\lib\site-packages\macad_gym\carla\multi_env.py", line 475, in _init_server                                                                                                                            live_carla_processes.add(os.getpgid(self._server_process.pid))                                                      
AttributeError: module 'os' has no attribute 'getpgid'

I think this is because the os library does not support the getpgid function in windows. Are they any workarounds for this issue?

[praveen-palanisamy]

Hi @eerkaijun ,
You are right. The issue is because the Python os module for Windows doesn't have support for getpgid and a few other (UNIX-specific) OS commands.
One workaround is to ask subprocess.Popen to create a new process group (wherein a new process group is created with the process ID as the group ID) which can then be killed/exited programmatically on windows by sending a Ctrl + C or Ctrl + Break event.

In summary, the changes for Windows would be to:

1. Add creationflags=subprocess.CREATE_NEW_PROCESS_GROUP as a new argument and remove the preexec_fn=os.setsid argument in the following Popen lines:

macad-gym/src/macad_gym/carla/multi_env.py

Lines 407 to 419 in 38884ac

	self._server_process = subprocess.Popen(
	("DISPLAY=:8 vglrun -d :7.{} {} {} -benchmark -fps=20"
	" -carla-server -world-port={}"
	" -carla-streaming-port=0".format(
	min_index,
	SERVER_BINARY,
	self._server_map,
	self._server_port,
	)),
	shell=True,
	preexec_fn=os.setsid,
	stdout=open(log_file, "w"),
	)

macad-gym/src/macad_gym/carla/multi_env.py

Lines 423 to 435 in 38884ac

	self._server_process = subprocess.Popen(
	("SDL_VIDEODRIVER=offscreen SDL_HINT_CUDA_DEVICE={}"
	" {} {} -benchmark -fps=20 -carla-server"
	" -world-port={} -carla-streaming-port=0".format(
	min_index,
	SERVER_BINARY,
	self._server_map,
	self._server_port,
	)),
	shell=True,
	preexec_fn=os.setsid,
	stdout=open(log_file, "w"),
	)

2. Then, in the cleanup() method listed below:
   

   macad-gym/src/macad_gym/carla/multi_env.py

   Lines 203 to 206 in 38884ac

   	def cleanup():
   	print("Killing live carla processes", live_carla_processes)
   	for pgid in live_carla_processes:
   	os.killpg(pgid, signal.SIGKILL)

Send pgid.send_signal(signal.CTRL_BREAK_EVENT)

Hope the above 2-step workaround works out for you.

[eerkaijun]

Thanks so much for the prompt response! I added the lines that you mentioned, but now I'm getting this error:

Initializing new Carla server...                                                                                        
FATAL ERROR while launching server: <class 'AttributeError'>

[praveen-palanisamy]

Looks like the error message is clipped/incomplete? Could you please post the full error log or let me know which line cause this error on your end?
Please post the contents of the server_*.log file as well for any additional info.

[eerkaijun]

The error message is not clipped, there's no more lines except the Fatal ERROR line. From the logs, I can see this, which seems to be an issue with the os library in windows again:

Traceback (most recent call last):
  File "C:\Eer Kai Jun\Autonomous Driving\env\lib\site-packages\macad_gym\carla\multi_env.py", line 221, in <module>
    from ray.rllib.env import MultiAgentEnv
ModuleNotFoundError: No module named 'ray'
DEBUG:macad_gym.carla.multi_env:module 'os' has no attribute 'setsid'

[praveen-palanisamy]

Ohh. Okay. Yeah, the setsid is also not available on Windows. I unfortunately don't yet have a Windows machine setup to test this out now.
The solution is to remove the preexec_fn=os.setsid argument to Popen. I have update my comment above(#32 (comment)) to include this change as well for Windows support.
Please give it a try with this change and post what you find.

[eerkaijun]

Hi @praveen-palanisamy, yeap that solves the problem! Thanks a lot for the help!