SDK v4 breaks Vuforia in landscape mode

[Windwoes]

The title pretty much says it all. It was not mentioned in the changelog, but in SDK v4.0, the Vuforia library has been updated. This seems to have helped the problem of the camera being dark in low light - which is good - but it also breaks Vuforia in landscape mode. Have a look at the following screenshots. Please note that the problem in SDK v4 is not just the image being skewed - as far as I can tell, it also screws up the positional lock as well (the lock marker will drift off the picture when moving the phone around)

SDK v3.5

SDK v4.0

[shelbyRobots]

I have noticed similar results. I copied the ConceptVuforiaNavRoverRuckus.java file and ran it. If you start holding the RC in portrait and then turn it to landscape and back, you will see the preview area change size to a long skinny aspect ratio.
On a loosely associated topic, would it be possible to move the network and robot status text to the black bar - placed just below the RC phone ID? Perhaps git rid of the "Active Configuration" label in the red included header bar and put the opmode there? Also possibly relocate the gamepad 1/2 text (even laid over the image like the error text). This allows for a larger area for the camera preview to display, and helps with aspect ratios. I have tested some of this, and it works with both Vuforia and OpenCV.

[Windwoes]

@shelbyRobots we need to wait till the forums open to be sure, but I'm pretty sure it's allowed to modify the GUI layout file however you like. FTAs might not want to assist you if you have problems, but honestly now that there's no longer a break between auto and tele, there's not really anything an FTA can do that you can't, at least for experienced teams.

[KevinEmery]

@shelbyRobots I would be wary of RS06 when talking about modifying the Driver Station app, and whether or not the modifications violate it being the "official" app. I'd definitely ask on the forums.

<RS06> Driver Station App - Teams must install the official “FTC Driver Station” app onto their Driver Station Android Device and use this app to control their Robot during a match. The Driver Station software version number must match the version number of the Robot Controller app.

[AlecHub]

The SDK is concerning the Robot Controller App. So <RS06> does not apply here.

<RS9> Software Modification - Teams are not allowed to modify the FIRST Tech Challenge Driver Station application or Robot Controller SDK in any fashion.

[gearsincorg]

Some points:

1. The Vuforia preview is displayed on the Robot Controller, so any modifications of the GUI would be done there, which is permitted.

2. As a rule, Vuforia will produce unexpected results if you rotate the camera AFTER it has initialized. It's always been this way. On a real robot this should never occur if the camera is hard mounted. When testing, just start the OpMode with the phone in the orientation you intend to run it on the robot.

[Windwoes]

@gearsincorg both of those points are unrelated to the issue that prompted me to open this issue, though

[shelbyRobots]

Thanks for the responses.
@FROGbots-4634
I wonder if #2 is really unrelated. I have seen the skewing (or perhaps scaling in Y) that your example shows. On a subsequent run, it will be gone. So maybe it has to do with a change in the view surface during layout that is similar in concept to what happens on a layout update caused by a rotation change.
As far as #1 -
I understand about being allowed to change the layout on the RC. I just try to avoid changing anything outside of TeamCode other than perhaps gradle files. I was making a suggestion that I thought might benefit other teams if the change was made in the upstream layout file. It doesn't look like the real estate in the top bar is being used for anything, so this is a simple change. As for the gamepad text - I am not crazy about it living on top of the image, but most of the time we are looking at the preview in auton, so the gamepads aren't being used.

[gearsincorg]

I'm checking this out.

One thing I noticed is that you are letting your phone auto rotate.
I typically have my phone locked into portrait mode, so when you run the Vuforia OpMode you get a much better viewing window in the landscape position.. I'm not saying you should Have to do this, but in the interim at may make your testing easier (and more predictable).

[gearsincorg]

What I can tell so far is that when the screen is in landscape mode, the vertical pixel scaling of the preview image is incorrect (too large). This explains why the target center seems to drift as you move the camera.

The RGB lines are being displayed in the correct positions, but the image is is sliding too far vertically as you move the camera. You can verify this by only moving the camera left and right... the target tracks correctly. But if you move it up and down, the image moves too far and leaves the axes lines behind.

The good news is that I believe the tracking is working correctly, and it's only a display problem. Again, you can get a consistent preview if you lock the screen in portrait mode.

I've passed this problem along to the developer who did much of the video/tracking integration.

[Windwoes]

@gearsincorg but, if you lock the phone in portrait and then put the phone in the landscape position, then wouldn't the angle and displacement data from Vuforia need to be adjusted to account for the 90 degree rotation? I'm planning to just modify the UI so that the camera preview fills the entire screen and all the text is just overlayed on top.

Good to hear that's it's just a visual glitch, although I will say that it makes it impossible to tell how well you're tracking the target if the axis drifts off the target, lol.

[lizlooney]

When you create the vuforia parameters, are you setting fillCameraMonitorViewParent to true?

[gearsincorg]

if you lock the phone in portrait and then put the phone in the landscape position, then wouldn't the angle and displacement data from Vuforia need to be adjusted to account for the 90 degree rotation?

Yes, but it already rotated is in the sample.

I ran the camera on it's side, (locked in portrait), for velocity vortex, and the axes responded as expected for target tracking. My experience is that you will get the desired results if the phone orientation you defined in your opmode, matches the phone orientation on the robot (with no auto rotation).

In the samples, the camera "orientation" includes a rotation to lay the camera on it's side.

Whatever you do, it's always wise to do a sanity check to ensure that the axes are responding as expected. I always do this sanity check each time I move/re orient the camera.

[AlecHub]

1. The Vuforia preview is displayed on the Robot Controller, so any modifications of the GUI would be done there, which is permitted.

Hi Phil, fyi <RS9> is a new rule for the Rover Ruckus season. Beginning this season, teams may not modify the Robot Controller SDK in any fashion.

Modifications to the RC GUI can be viewed as a modification of the Robot Controller SDK.

Because of the new <RS9> rule, and because nowhere in the rules does it say modifications to the RC GUI is permitted, I suggest that teams get any modification to the RC approved by the GDC, via the Game Q&A Forum, to avoid issues at Robot/Field inspections, and to avoid issues with the Head Ref during matches.

[gearsincorg]

Wow. How did I miss that one. I’m surprised. I’ll need to re-read in detail to clarify my thinking.

--

Phil Malone GEARS Inc. phil.malone@mr-phil.com Office: 301.387.2331 Mobile: 301.501.7424

[shelbyRobots]

But what does modifying the Robot Controller SDK mean under ?
It obviously doesn't mean that you can't touch anything delivered as part of the SDK zip from github - TeamCode folder is part of the zip, but we have to add to it. Likewise, gradle files seem like fair game - we have to modify those to add libs, change compile versions, etc.
I would lean towards not being able to modify anything that is delivered as part of the .aar files. But we obviously need a judgement from those on high.
Note that the layout can be changed completely programmatically inside TeamCode - without touching the layout xml file.
We really need the Q&A forums to be opened - we are now less than 6 weeks from our first qualifier.