Linux RolloutWorkers: screenshot format

[iar1000]

Hi, I am working on an extension to enable the RolloutWorkers run on Linux. I have an interface that let's me run and steer Trackmania in Lutris, but when I run the provided agent on the test-track, with the given weights, the car is not able to finish the track.
It could be that the CNN is confused, as I am not sure what the function WindowInterface.screenshot() is returning. I had to make assumptions (returns with default settings a 128x256x4 np array, RGB and opacity), since I can't confirm it on a Windows machine. I hope you can help me out with following two questions:

q1: With the given weights and the test setup, the car should be able to succeed?
q2: what is the format of img that is being returned by screenshot? (np.array of dim ..., whereas arr[0] is ...)

Thanks a lot,

p

[yannbouteiller]

Hi, it should be able to complete the tmrl-test track yes, unless I messed things up. I'll try to give you more info about the format returned by screenshots on Windows soon, I don't have my PC atm.

[yannbouteiller]

Hi again.

Here is a pickled output of the screenshot() function in Windows. It is a numpy array of shape (128, 256, 4), After unzipping, you can open it with the following code snippet:

import pickle as pkl
import cv2

with open("screenshot.pkl", "rb") as f:
    snap = pkl.load(f)
print(snap.shape)
cv2.imshow("test", snap)
cv2.waitKey(0)
cv2.destroyAllWindows()

screenshot.zip

[yannbouteiller]

Note that the game window is previously resized by win32ui to 128 {height} * 256 (width), which is the minimum on Windows.

[yannbouteiller]

Also, I confirm that the default CNN should complete the tmrl-test track. It rams into walls a little bit but that is part of the policy it has learnt, Note also that it has been trained with the canadian skin, I am not sure how it handles other skins, since it is trained in 3rd-person view.