recreating simulink schemes for ubuntu 20.4.

[StefanJakubjak]

Hi, i would like to create simulink blocks from your quanser openai driver. I know there are already block schemes for simulink from quanser: https://quanserinc.box.com/shared/static/08vjgurb59omat6s1xd9u42m1xhko199.zip
Quarc is needed to run those and since quarc is not available on ubuntu I would like to recreate those for simulink with quanser openai driver. I already sucessfully run 1 of yours examples as Sfunction from matlab with python:
import gym
from gym_brt import QubeSwingupEnv

num_episodes = 10
num_steps = 250

with QubeSwingupEnv() as env:
for episode in range(num_episodes):
state = env.reset()
for step in range(num_steps):
action = env.action_space.sample()
state, reward, done, _ = env.step(action)

How can i create and run basic experiments like those examples in hil sdk written in c using your driver? Is it even possible? I couldn't find any basic example like those in hil sdk.

[polzounov]

Hi Stephan,

This library is focused on Reinforcement Learning which is why the default interface is an OpenAI gym environment.
There is a lower-level interface to the Qube (and the Quanser Aero) in quanser_wrapper.pyx

For example if you want to apply a random voltage at every timestep:

import random
from gym_brt.quanser.quanser_wrapper.quanser_wrapper import QubeServo2

with QubeServo2(frequency=250, max_voltage=3.0) as qube:
    while True:
            voltages = [random.gauss(mu=0, sigma=1.0)]
            currents, encoders, others = qube.action(voltages)
            print(f"voltage={voltage}, currents={currents}, encoders={encoders}, others={others}")

Or if you want to apply a sine wave:

import math
from gym_brt.quanser.quanser_wrapper.quanser_wrapper import QubeServo2

with QubeServo2(frequency=250, max_voltage=3.0) as qube:
    count = 1
    sine_wave_period = 2  # in seconds
    while True:
        voltages = [math.sin(count / (frequency * sine_wave_period))]
        currents, encoders, others = qube.action(voltages)
        print(f"voltage={voltage}, currents={currents}, encoders={encoders}, others={others}")

        count += 1

[StefanJakubjak]

Thank you very much for answer, i will try it. :)

[StefanJakubjak]

So i have tried it and it's working but i would like to do something like:
hil_read_analog(board, channels)
hil_write_analog(board, channels, voltages)
in quanser hil api you can do :
result=hil_read(board, analog_channels, NUM_ANALOG_CHANNELS, encoder_channels, NUM_ENCODER_CHANNEL, digital_channels,NUM_DIGITAL_CHANNELS, other_channels, NUM_OTHER_CHANNELS, &voltages[0], &counts[0], &states[0], &values[0] );
Is it possible ?

[StefanJakubjak]

Can i do something like that ?

[polzounov]

Hi Stefan,

The way this library is implemented is by abstracting away some of the lower-level information like those used in the HIL SDK examples. So that you can use the Qube but not need to deal with those HIL API functions.

You can make this work by looking at the code in quanser_wrapper.pyx as a starting point and using that to create Cython functions that can then be called from Python. If you would like to add some additional functionality similar to that feel free to open up a pull request

I've been looking into rewriting the library with ctypes which would have made it easier to do exactly that but have not had much time or need for those changes on the reinforcement learning side.

[StefanJakubjak]

Thank you very much, I know how to code c and Python but i don't really know how to that sorry. I also found that quanser has python api, almost same as c api for linux. You have to install quarc runtime and than you can install this python api on ubuntu.