Skip to content

karma/jamsine plot interaction tests #61

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Dec 2, 2015
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions circle.yml
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
general:
artifacts:
- "build/test_images/" # relative to the build directory
- "build/test_images_diff/" # relative to the build directory
- "build/test_images/"
- "build/test_images_diff/"

machine:
node:
version: 4.2.1
version: 4.2.1
services:
- docker


dependencies:
pre:
pre:
- docker pull plotly/imageserver:latest
post:
- npm run cibuild
Expand Down
17 changes: 3 additions & 14 deletions tasks/baseline.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,23 +2,12 @@
#
# TODO adapt this for Windows
#
# TODO add package.json config arguments to configure:
# - container name,
# - ports and
# - imageserver version
#
# ===============================================================================

CONTAINER_NAME="imagetest"
IMAGE_NAME="registry.plot.ly:5000/imageserver"
IMAGE_VERSION="1.3.0"
CONTAINER_NAME="imagetest" # same as in docker-compose.yml

# Run docker container:
#
# docker run -d --name $CONTAINER_NAME \
# -v $PWD/plotly.js:/var/www/streambed/image_server/plotly.js \
# -p 9010:9010 -p 2022:22 \
# $IMAGE_NAME:[$IMAGE_VERSION]
# create/run/start docker container with:
# $ docker-compose up -d

CMD=(
"cd /var/www/streambed/image_server/plotly.js &&"
Expand Down
17 changes: 3 additions & 14 deletions tasks/test_image.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,23 +2,12 @@
#
# TODO adapt this for Windows
#
# TODO add package.json config arguments to configure:
# - container name,
# - ports and
# - imageserver version
#
# ===============================================================================

CONTAINER_NAME="imagetest"
IMAGE_NAME="registry.plot.ly:5000/imageserver"
IMAGE_VERSION="1.3.0"
CONTAINER_NAME="imagetest" # same as in docker-compose.yml

# Run docker container:
#
# docker run -d --name $CONTAINER_NAME \
# -v $PWD/plotly.js:/var/www/streambed/image_server/plotly.js \
# -p 9010:9010 -p 2022:22 \
# $IMAGE_NAME:[$IMAGE_VERSION]
# create/run/start docker container with:
# $ docker-compose up -d

CMD=(
"cd /var/www/streambed/image_server/plotly.js &&"
Expand Down
5 changes: 4 additions & 1 deletion test/image/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,9 @@ docker images
docker ps -a
```

whereas `docker ps` lists only the started containers.


### Stop your testing container

Inside your `plotly.js` directory, run
Expand All @@ -87,4 +90,4 @@ Inside your `plotly.js` directory, run
docker-compose rm -f
```

For more comprehensive information about docker, please refer to [docker document](http://docs.docker.com/)
For more comprehensive information about docker, please refer to the [docker docs](http://docs.docker.com/).
44 changes: 23 additions & 21 deletions test/image/compare_pixels_test.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,21 @@ else runSingle(userFileName);
function runAll () {
test('testing mocks', function (t) {

console.error('### beginning pixel comparison tests ###');
var files = fs.readdirSync(constants.pathToTestImageMocks);

// -1 for font-wishlist and
// -38 for the gl2d mocks
t.plan(files.length - 39);
/*
* Some test cases exhibit run-to-run randomness;
* skip over these few test cases for now.
*
* More info:
* https://github.com/plotly/plotly.js/issues/62
*
* 40 test cases are removed:
* - font-wishlist (1 test case)
* - all gl2d (38)
* - gl2d_bunny-hull (1)
*/
t.plan(files.length - 40);

for (var i = 0; i < files.length; i ++) {
testMock(files[i], t);
Expand All @@ -54,6 +63,9 @@ function testMock (fileName, t) {
// TODO fix race condition in gl2d image generation
if(fileName.indexOf('gl2d_') !== -1) return;

// TODO fix run-to-run randomness
if(fileName === 'gl3d_bunny-hull.json') return;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this particular 3d mock fails in ~ 1 out of 10 attempts.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we make an issue for this and embed the issue number in this comment?


var figure = require(path.join(constants.pathToTestImageMocks, fileName));
var bodyMock = {
figure: figure,
Expand All @@ -66,7 +78,6 @@ function testMock (fileName, t) {
var diffPath = path.join(constants.pathToTestImagesDiff, 'diff-' + imageFileName);
var savedImageStream = fs.createWriteStream(savedImagePath);
var options = getOptions(bodyMock, 'http://localhost:9010/');
var statusCode;

function checkImage () {
var options = {
Expand All @@ -75,17 +86,12 @@ function testMock (fileName, t) {
tolerance: 0.0
};

if(statusCode === 485) {
console.error(imageFileName, '- skip');
}
else {
gm.compare(
savedImagePath,
path.join(constants.pathToTestImageBaselines, imageFileName),
options,
onEqualityCheck
);
}
gm.compare(
savedImagePath,
path.join(constants.pathToTestImageBaselines, imageFileName),
options,
onEqualityCheck
);
}

function onEqualityCheck (err, isEqual) {
Expand All @@ -95,16 +101,12 @@ function testMock (fileName, t) {
}
if (isEqual) {
fs.unlinkSync(diffPath);
console.error(imageFileName + ' is pixel perfect');
}

t.ok(isEqual, savedImagePath + ' should be pixel perfect');
t.ok(isEqual, imageFileName + ' should be pixel perfect');
}

request(options)
.on('response', function(response) {
statusCode = response.statusCode;
})
.pipe(savedImageStream)
.on('close', checkImage);
}
8 changes: 8 additions & 0 deletions test/jasmine/assets/create_graph_div.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
'use strict';

module.exports = function createGraphDiv() {
var gd = document.createElement('div');
gd.id = 'graph';
document.body.appendChild(gd);
return gd;
};
6 changes: 6 additions & 0 deletions test/jasmine/assets/destroy_graph_div.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
'use strict';

module.exports = function destroyGraphDiv() {
var gd = document.getElementById('graph');
document.body.removeChild(gd);
};
9 changes: 9 additions & 0 deletions test/jasmine/karma.ciconf.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,15 @@ function func(config) {
func.defaultConfig.logLevel = config.LOG_INFO;

// Continuous Integration mode

/*
* WebGL interaction test cases fail on the CircleCI
* most likely due to a WebGL/driver issue;
* exclude them from the CircleCI test bundle.
*
*/
func.defaultConfig.exclude = ['tests/gl_plot_interact_test.js'];

// if true, Karma captures browsers, runs the tests and exits
func.defaultConfig.singleRun = true;

Expand Down
46 changes: 46 additions & 0 deletions test/jasmine/tests/gl_plot_interact_test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
var d3 = require('d3');

var Plotly = require('@src/index');

var createGraphDiv = require('../assets/create_graph_div');
var destroyGraphDiv = require('../assets/destroy_graph_div');

/*
* WebGL interaction test cases fail on the CircleCI
* most likely due to a WebGL/driver issue
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah gotcha, perhaps we can put this comment in the karma.ciconf.js file

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the same issue with the bunny-hill test? If it is can we link all these to an issue?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the same issue with the bunny-hill test?

No. We can't have WebGL karma tests at the moment because karma fails to render <canvas> in the CircleCI box.

I'm not sure what's special about the bunny hull test case that makes output random pixel in 1 out of 10 runs. Maybe increasing the delay on snapshot can fix it.

*
*/


describe('Test plot structure', function () {
'use strict';

afterEach(destroyGraphDiv);

describe('gl3d plots', function() {
var mock = require('@mocks/gl3d_marker-arrays.json');

beforeEach(function(done) {
Plotly.plot(createGraphDiv(), mock.data, mock.layout).then(done);
});

it('has one *canvas* node', function() {
var nodes = d3.selectAll('canvas');
expect(nodes[0].length).toEqual(1);
});
});

describe('gl2d plots', function() {
var mock = require('@mocks/gl2d_10.json');

beforeEach(function(done) {
Plotly.plot(createGraphDiv(), mock.data, mock.layout).then(done);
});

it('has one *canvas* node', function() {
var nodes = d3.selectAll('canvas');
expect(nodes[0].length).toEqual(1);
});
});

});
Loading