IMDB: Update import script, source files and dump (#12)

* Update script and source files
- Removed duplicate edges
- Removed empty fields ("")
- Removed a dummy edge
- Converted releaseDate to number, corrected timezone (was apparently assuming Europe/Berlin instead of UTC)
- Fixed 13 releaseDates manually
- Make the script work with ArangoDB 3.7/3.8
- Don't drop edge _key anymore (duplicates removed in source file)
- Add type: "Genre" to genre vertices
- Disable creation of fulltext indexes
* Update README
* Change source file extensions from .json to .jsonl
* Update dump

Author: Simran-B

Graphs/IMDB/README.md

@@ -1,28 +1,59 @@
-This dataset is taken from the IMDB service http://www.imdb.com.
+# IMDB Movies and Actors Graph Dataset
 
-Before you start please make sure you do not have collections named:
-**imdb_vertices**, **imdb_edges**
-and you do not have a graph called
-**imdb** 
-These will be overwritten by the importer.
+This dataset is taken from the Internet Movie Database <http://www.imdb.com>.
 
+The dataset has two collections:
+- **imdb_vertices**:
+  Containing all Movies, Actors, Directors etc. and Genres of movies that can
+  be used for traversal.
+- **imdb_edges**:
+  Containing the relations between the vertices, who-acts-where,
+  movie-has-genre etc.
 
-To import the data execute the following on your bash:
+Furthermore, it comes with a named graph **imdb** using both collections.
 
-```Bash
+## Remarks
+
+The attributes `birthday` and `lastModified` are Unix timestamps in milliseconds,
+but they are stored as strings. `releaseDate` is stored as number.
+
+## Restore from dump
+
+To restore the data from the dump into a new database `IMDB`, execute the
+following on your bash:
+
+```bash
+ unix> arangorestore --server.endpoint tcp://<host>:<port> --server.database IMDB --create-database --include-system-collections
+```
+
+To restore the dump with default settings you may run the following instead:
+
+```bash
  unix> ./import.sh
 ```
 
-You will then have the following two collections:
-* **imdb_vertices** Containing all Movies, Actors, Directores etc. and Genres of movies that can be used for traversal.
-* **imdb_edges** Containing the relations between the vertices, who-acts-where, movie-has-genre etc.
+## Import from source files
+
+Before you start, please make sure you do not have **collections** named:
+- **imdb_vertices**
+- **imdb_edges**
+and that you do not have a **graph** called:
+- **imdb**
+
+These will be overwritten by the importer!
+
+To import the data from the `nodes.json` and `edges.json` source files into the
+`_system` collection, run this on your bash:
+
+```bash
+ unix> arangosh --server.endpoint tcp://<host>:<port> --javascript.execute ./import.js
+```
 
-Furthermore you have the graph **imdb** using both collections.
+## Create new dump
 
-The above import uses dumps of the collections. In order to recreate the collections from
-the source file `hero-comic-network.csv`, you can use
+To create a new dump of the imported data including the two collections and
+the graph, run:
 
-```Bash
- unix> cat import.js | arangosh
- unix> arangodump --collection imdb_edges --collection imdb_vertices
+```bash
+ unix> arangodump --server.endpoint tcp://<host>:<port> --server.database <database> --include-system-collections --collection _graphs --collection imdb_vertices --collection imdb_edges --compress-output false --envelope true
 ```

Graphs/IMDB/dump/ENCRYPTION

@@ -0,0 +1 @@
+none

Graphs/IMDB/dump/_graphs_c827636f2b54efb49f1f02feeeacfb01.data.json

@@ -0,0 +1 @@
+{"type":2300,"data":{"_key":"imdb","_id":"_graphs/imdb","_rev":"_cPlZM_e---","edgeDefinitions":[{"collection":"imdb_edges","from":["imdb_vertices"],"to":["imdb_vertices"]}],"orphanCollections":[]}}

Graphs/IMDB/dump/_graphs_c827636f2b54efb49f1f02feeeacfb01.structure.json

@@ -0,0 +1 @@
+{"indexes":[],"parameters":{"allowUserKeys":true,"cacheEnabled":false,"cid":"163325","deleted":false,"globallyUniqueId":"_graphs","id":"163325","isDisjoint":false,"isSmart":false,"isSmartChild":false,"isSystem":true,"keyOptions":{"allowUserKeys":true,"type":"traditional","lastValue":0},"minReplicationFactor":1,"minRevision":"0","name":"_graphs","numberOfShards":1,"planId":"163325","replicationFactor":1,"schema":null,"shardKeys":["_key"],"shards":{},"status":3,"syncByRevision":true,"type":2,"usesRevisionsAsDocumentIds":true,"version":9,"waitForSync":false,"writeConcern":1}}

Graphs/IMDB/dump/dump.json

@@ -0,0 +1 @@
+{"database":"IMDB","lastTickAtDumpStart":"356559","useEnvelope":true,"properties":{"id":"163323","name":"IMDB","isSystem":false}}

Graphs/IMDB/dump/imdb_edges.data.json

@@ -0,0 +1 @@
+{"indexes":[],"parameters":{"allowUserKeys":true,"cacheEnabled":false,"cid":"163369","deleted":false,"globallyUniqueId":"hABC5EBAFC11E/163369","id":"163369","isDisjoint":false,"isSmart":false,"isSmartChild":false,"isSystem":false,"keyOptions":{"allowUserKeys":true,"type":"traditional","lastValue":249839},"minReplicationFactor":1,"minRevision":"_cPlZM-u---","name":"imdb_edges","numberOfShards":1,"planId":"163369","replicationFactor":1,"schema":null,"shardKeys":["_key"],"shards":{},"status":3,"syncByRevision":true,"type":3,"usesRevisionsAsDocumentIds":true,"version":9,"waitForSync":false,"writeConcern":1}}