Skip to content

OpenStreetMapColombia/yourmapswipe-nodejs-workers

Repository files navigation

State: Early Open Sourced Version (Please report bugs)

This repo contains 2 applications: The first one are the workers, the second one is the Mapswipe importer. Both run periodically.

Note: We are aware that set-up is complicated right now, and we are raising funds for a stage 2 of Mapswipe (since this was all developed during a pilot phase), where we will have automated deployments. If you wish to contribute to this, please contact pete.masters@london.msf.org.

#Add the following to your rc.local file on linux to make it auto start if your server restarts:

cd PATH_YOU_PUT_THIS
sh run-importer.sh || /bin/true
sh run-worker.sh || /bin/true

#Requirements:

  • Make sure you have created a Firebase project on firebase.google.com
  • Make sure you have a MySQL database with the following tables:
CREATE TABLE `projects` (
   `project_id` int(11) NOT NULL,
   `objective` varchar(20) DEFAULT NULL,
   `name` varchar(45) DEFAULT NULL,
   PRIMARY KEY (`project_id`)
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8;


 CREATE TABLE `results` (
   `task_id` varchar(45) NOT NULL,
   `user_id` varchar(45) NOT NULL,
   `project_id` int(5) NOT NULL,
   `timestamp` bigint(32) NOT NULL,
   `result` int(1) NOT NULL,
   `wkt` varchar(256) DEFAULT NULL,
   `task_x` varchar(45) NOT NULL,
   `task_y` varchar(45) NOT NULL,
   `task_z` varchar(45) NOT NULL,
   `duplicates` int(5) DEFAULT '0',
   PRIMARY KEY (`task_id`,`user_id`,`project_id`)
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8;


###Install:

  • Make sure you have the Node.js installed
  • Set up an nginx server and place the root on /var/www/html, create that directory if it doesn't exist yet, and make it writable for the user that will run this application. This is where your API will become available.
  • Follow the steps here: https://firebase.google.com/docs/server/setup
  • Also enable firebase auth so users can authenticate in the app
  • Add the new service account .json file in cfg/your-service-file.json. Do a search in the project for your-service-file.json and replace the name if you want to use the original file name.
  • Do a search in the project for YOUR-DOMAIN and replace all the occurances with your firebase domain. It's very important that you get ALL of them.
  • Do a search in the project for YOUR-MYSQL and replace all the occurances with your database credentials
  • In api_key.txt place your bing API key
  • Now go to sample_importer/import.html, edit the firebase credentials to match your firebase objects, and after that's done, open the file in a browser, and import your first project!
  • Don't forget to set up rules for the classes in firebase. Should be like this:
  "rules": {
    ".read": false,
    ".write": "auth != null",
      "groups" : {
        ".write": false,
        ".read" : true,
        "$project_id" : {
          "$group_id" : {
            "completedCount" : {
              ".write": "auth != null",
            }
          },
        ".indexOn": ["distributedCount", "completedCount"]
        }
      },
      "imports" : {
        ".read" : true,
        ".write" : true
      },
     "projects" : {
        ".write": false,
        ".read" : true,
      },
    "results" : {
      ".write": false,
      ".read" : true,
        "$task_id" : {
          "$user_id" : {
          ".write": "auth != null && auth.uid == $user_id"
          }
        }
      },
      "users": {
      "$uid": {
        ".read": "auth != null && auth.uid == $uid",
        ".write": "auth != null && auth.uid == $uid",
      }
    }
  }
}
  • Run the the importer (make sure you have plenty of RAM available)
./run-importer.sh
  • Then finally, run the workers that count contributions etc.
./run-worker.sh 

API Documentation for generated files in /var/www/html available on: https://docs.google.com/document/d/1RwN4BNhgMT5Nj9EWYRBWxIZck5iaawg9i_5FdAAderw/edit#heading=h.wp1a8ue6nwhv

  • If you want to allocate less memory (because you have smaller tasks) you can edit the --max-old-space-size=SIZE_IN_MB_YOU_NEED values

#Contributing Contributions are welcome. Please fork the repository, test incredibly well on your end, and make a pull request once finished.

About

Backend workers for Mapswipe, so that you can create your own mapswipe version!

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors