Inital commit

This commit is contained in:
Sören Oesterwind 2022-03-06 18:36:36 +01:00
commit bedd69436b
175 changed files with 9965 additions and 0 deletions

3
.eslintignore Normal file
View File

@ -0,0 +1,3 @@
node_modules
dist
main.js

17
.eslintrc.json Normal file
View File

@ -0,0 +1,17 @@
{
"root": true,
"plugins": [
"@typescript-eslint"
],
"extends": [
"eslint:recommended",
"plugin:@typescript-eslint/eslint-recommended",
"plugin:@typescript-eslint/recommended"
],
"rules": {
"@typescript-eslint/no-var-requires": 0,
"no-control-regex": 0
}
}

7
.gitignore vendored Normal file
View File

@ -0,0 +1,7 @@
node_modules
package-lock.json
config/default.json
/main.js
static/css/tailwindInclude.css
FLAG
*/db_table_config.sql

71
LICENSE Normal file
View File

@ -0,0 +1,71 @@
GNU LESSER GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
This version of the GNU Lesser General Public License incorporates the terms and conditions of version 3 of the GNU General Public License, supplemented by the additional permissions listed below.
0. Additional Definitions.
As used herein, "this License" refers to version 3 of the GNU Lesser General Public License, and the "GNU GPL" refers to version 3 of the GNU General Public License.
"The Library" refers to a covered work governed by this License, other than an Application or a Combined Work as defined below.
An "Application" is any work that makes use of an interface provided by the Library, but which is not otherwise based on the Library. Defining a subclass of a class defined by the Library is deemed a mode of using an interface provided by the Library.
A "Combined Work" is a work produced by combining or linking an Application with the Library. The particular version of the Library with which the Combined Work was made is also called the "Linked Version".
The "Minimal Corresponding Source" for a Combined Work means the Corresponding Source for the Combined Work, excluding any source code for portions of the Combined Work that, considered in isolation, are based on the Application, and not on the Linked Version.
The "Corresponding Application Code" for a Combined Work means the object code and/or source code for the Application, including any data and utility programs needed for reproducing the Combined Work from the Application, but excluding the System Libraries of the Combined Work.
1. Exception to Section 3 of the GNU GPL.
You may convey a covered work under sections 3 and 4 of this License without being bound by section 3 of the GNU GPL.
2. Conveying Modified Versions.
If you modify a copy of the Library, and, in your modifications, a facility refers to a function or data to be supplied by an Application that uses the facility (other than as an argument passed when the facility is invoked), then you may convey a copy of the modified version:
a) under this License, provided that you make a good faith effort to ensure that, in the event an Application does not supply the function or data, the facility still operates, and performs whatever part of its purpose remains meaningful, or
b) under the GNU GPL, with none of the additional permissions of this License applicable to that copy.
3. Object Code Incorporating Material from Library Header Files.
The object code form of an Application may incorporate material from a header file that is part of the Library. You may convey such object code under terms of your choice, provided that, if the incorporated material is not limited to numerical parameters, data structure layouts and accessors, or small macros, inline functions and templates (ten or fewer lines in length), you do both of the following:
a) Give prominent notice with each copy of the object code that the Library is used in it and that the Library and its use are covered by this License.
b) Accompany the object code with a copy of the GNU GPL and this license document.
4. Combined Works.
You may convey a Combined Work under terms of your choice that, taken together, effectively do not restrict modification of the portions of the Library contained in the Combined Work and reverse engineering for debugging such modifications, if you also do each of the following:
a) Give prominent notice with each copy of the Combined Work that the Library is used in it and that the Library and its use are covered by this License.
b) Accompany the Combined Work with a copy of the GNU GPL and this license document.
c) For a Combined Work that displays copyright notices during execution, include the copyright notice for the Library among these notices, as well as a reference directing the user to the copies of the GNU GPL and this license document.
d) Do one of the following:
0) Convey the Minimal Corresponding Source under the terms of this License, and the Corresponding Application Code in a form suitable for, and under terms that permit, the user to recombine or relink the Application with a modified version of the Linked Version to produce a modified Combined Work, in the manner specified by section 6 of the GNU GPL for conveying Corresponding Source.
1) Use a suitable shared library mechanism for linking with the Library. A suitable mechanism is one that (a) uses at run time a copy of the Library already present on the user's computer system, and (b) will operate properly with a modified version of the Library that is interface-compatible with the Linked Version.
e) Provide Installation Information, but only if you would otherwise be required to provide such information under section 6 of the GNU GPL, and only to the extent that such information is necessary to install and execute a modified version of the Combined Work produced by recombining or relinking the Application with a modified version of the Linked Version. (If you use option 4d0, the Installation Information must accompany the Minimal Corresponding Source and Corresponding Application Code. If you use option 4d1, you must provide the Installation Information in the manner specified by section 6 of the GNU GPL for conveying Corresponding Source.)
5. Combined Libraries.
You may place library facilities that are a work based on the Library side by side in a single library together with other library facilities that are not Applications and are not covered by this License, and convey such a combined library under terms of your choice, if you do both of the following:
a) Accompany the combined library with a copy of the same work based on the Library, uncombined with any other library facilities, conveyed under the terms of this License.
b) Give prominent notice with the combined library that part of it is a work based on the Library, and explaining where to find the accompanying uncombined form of the same work.
6. Revised Versions of the GNU Lesser General Public License.
The Free Software Foundation may publish revised and/or new versions of the GNU Lesser General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the Library as you received it specifies that a certain numbered version of the GNU Lesser General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that published version or of any later version published by the Free Software Foundation. If the Library as you received it does not specify a version number of the GNU Lesser General Public License, you may choose any version of the GNU Lesser General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide whether future versions of the GNU Lesser General Public License shall
apply, that proxy's public statement of acceptance of any version is permanent authorization for you to choose that version for the Library.

82
README.MD Normal file
View File

@ -0,0 +1,82 @@
# Pointsight
Pointsight is a centrailzed geolocatlized api aggreagtor.
# To be done before feature lock
- [X] Feature to report points
- [ ] API usage tracking (MySQL done; influx done; limiting tbd)
- [X] Proper error messages, on page crash loading failure, lost connection
- [X] Shareable points, open by link
- [X] Offline version
- [X] Filter sidebar
# Configuration
This is the configuration with all it's defaults.
```
let jsonConfigGlobal = {
fontAwesome: undefined,
mapboxAccessToken: undefined,
cookieSecret: undefined,
mapquest: undefined,
here: undefined,
sentryDsn: undefined,
database: {
host: "127.0.0.1",
user: "pointsight",
password: "CHANGE_ME",
database: "pointsight",
customSettings: {
wait_timout: 28800
}
},
env: "PROD",
maint: true,
betaMode: false,
port: 3000,
adress: '127.0.0.1',
taxonomyCacheInterval: 5,
metrics: {
influx: {
enable: false,
host: "127.0.0.1",
database: "pointsight",
user: "pointsight",
password: "CHANGE_ME",
},
writeMetricsToMySQL: true
}
}
```
# How to deploy
1. Make sure Node.js is installed
2. Clone repo
3. Run the `npm install` command
4. Create the `config/default.json` file with proper settings
1. Make sure `env` is set to `PROD`
2. Make sure `maint` is set to `false`
3. Make sure the database credentials are set
5. Start the server with `npm start`, this starts the server on port 3000
## Troubleshooting
**The map stays empty**
Cause: The data cannot be loaded because the API key was not imported.
Solution: Import the `b03f8aaf-1f32-4d9e-914a-9a50f904833d` into the `apikeys` table. After importing it you may use the apikey manager to change it's options.
**The map is not being displayed**
Cause: The maps tiles cannot be loaded as the API key is missing.
Solution: Make sure the config contains the correct apikey.
# For developers
## Adding a new module
1. Copy the example module folder
2. Change the `main.js` to your needs.
1. Update the modules meta data in the `getModuleMeta()` funcion
2. Make sure to use the right country for your module

View File

@ -0,0 +1,2 @@
DROP TABLE IF EXISTS `airspy`
CREATE TABLE IF NOT EXISTS `airspy` ( `id` BIGINT unsigned NOT NULL AUTO_INCREMENT, `name` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `description` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `source_id` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `loc` POINT NOT NULL, `protocol_id` SMALLINT unsigned NOT NULL, `link` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, UNIQUE (id));

View File

@ -0,0 +1,13 @@
DROP TABLE IF EXISTS `airspy`
#NL
CREATE TABLE IF NOT EXISTS `airspy` (
`id` BIGINT unsigned NOT NULL AUTO_INCREMENT,
`name` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`description` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`source_id` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`loc` POINT NOT NULL,
`protocol_id` SMALLINT unsigned NOT NULL,
`link` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
UNIQUE (id)
);
#NL

View File

@ -0,0 +1,157 @@
const fs = require("fs");
const bent = require("bent");
const mysql = require("mysql");
const tools = require("../../functions");
let con = undefined;
function initialize(conection) {
con = conection;
// Prepare all of the tables needed
const array = fs
.readFileSync("./apiHandler/airspydirectory/db_table_config.sql")
.toString()
.split("\n");
for (i in array) {
con.query(array[i], function (err, result) {
tools.handleMysqlErrors(err, getModuleMeta().title);
console.log("Table created for AirspyDirectory");
});
}
const airspyDirectoryApiUrl = "https://airspy.com/directory/status.json";
const mysqlQuery =
"INSERT INTO airspy (name, description, loc, source_id, protocol_id, link) VALUES ( ?, ?, POINT(?,?), ?, ?, ?)";
// Request API and save into table
const request = bent(airspyDirectoryApiUrl, "GET", "json", 200);
request().then(function(body){
console.log("[AirSpyDirectory] Request done");
for (let c = 0; c < body.servers.length; c++) {
const elm = body.servers[c];
con.query(
mysqlQuery,
[
elm.deviceType,
elm.generalDescription +
"<br> Adress: " +
elm.streamingHost +
":" +
elm.streamingPort +
"<br>" +
"Owned by: " +
elm.ownerName,
elm.antennaLocation.lat,
elm.antennaLocation.long,
0,
0,
"https://airspy.com/directory/status.json",
],
function (err, result) {
tools.handleMysqlErrors(err, getModuleMeta().title);
}
);
}
console.log("[AirSpyDirectory] Insert into Database done!");
},
function (err) {
console.warn(
"[" +
getModuleMeta().title +
"] Was unable to resolve request with error: " +
err
);
});
}
function queryData(boundNorthEastLatLng, boundSouthWestLatLng, minimal) {
// Create a new promise to return, DBs are slow.
return new Promise(function (resolve, reject) {
// The Promise constructor should catch any errors thrown on
// this tick. Alternately, try/catch and reject(err) on catch.
let elementsToQuery = "*";
if (minimal) {
elementsToQuery = "id,loc,source_id";
}
// This is taken from https://stackoverflow.com/questions/21208697/select-all-geospatial-points-inside-a-bounding-box
const sqlQuery = " \
SELECT " + elementsToQuery + " \
FROM airspy \
WHERE MBRContains( \
GeomFromText( 'LINESTRING(? ?,? ?)' ), \
loc)";
con.query(sqlQuery, [boundNorthEastLatLng.lat, boundNorthEastLatLng.lng, boundSouthWestLatLng.lat, boundSouthWestLatLng.lng], function (err, rows, fields) {
// Call reject on error states,
// call resolve with results
if (err) {
return reject(err); // Probably bad for PROD
}
const results = [];
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e];
let element = {
type: "general-poi",
source: "AirSpyDirectory",
titel: currentRow.name,
description: currentRow.description,
url: currentRow.link,
uid: "airspydirectory_" + currentRow.id,
location: currentRow.loc,
icon: "signalIcon",
};
if (minimal) {
const stripableElements = ["title", "description", "url", "type"];
for (h in stripableElements) {
delete element[stripableElements[h]];
}
}
results.push(element);
}
resolve(results);
});
});
}
function queryItem(uid) {
const uidParts = uid.split("_");
const sqlQuery = "SELECT * FROM airspy WHERE id=?";
return new Promise(function (resolve, reject) {
con.query(sqlQuery, [parseInt(uidParts[1])], function (err, rows, fields) {
if (err) {
return reject(err);
}
const results = [];
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e];
const element = {
type: "general-poi",
source: "AirSpyDirectory",
titel: currentRow.name,
description: currentRow.description,
url: currentRow.link,
uid: "airspydirectory_" + currentRow.id,
location: currentRow.loc,
icon: "signalIcon",
};
results.push(element);
}
resolve(results);
});
});
}
function getModuleMeta() {
return {
title: "AirSpyDirectory",
exactName: "airspydirectory",
country: ["*"],
tags: ["RF"],
features: ["tags"],
limited: false,
};
}
module.exports = { initialize, queryData, getModuleMeta, queryItem };

View File

@ -0,0 +1,4 @@
DROP TABLE IF EXISTS `autobahn`
DROP TABLE IF EXISTS `source`
CREATE TABLE IF NOT EXISTS `source` ( `source_id` INT unsigned NOT NULL AUTO_INCREMENT, `source_name` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `source_live` BIT NOT NULL, `source_link` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `source_type` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `source_area` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, UNIQUE (source_id));
CREATE TABLE IF NOT EXISTS `autobahn` ( `id` BIGINT unsigned NOT NULL AUTO_INCREMENT, `name` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `source_id` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `loc` POINT NOT NULL, `protocol_id` SMALLINT unsigned NOT NULL, `link` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, UNIQUE (id));

View File

@ -0,0 +1,24 @@
DROP TABLE IF EXISTS `autobahn`
#NL
DROP TABLE IF EXISTS `source`
#NL
CREATE TABLE IF NOT EXISTS `source` (
`source_id` INT unsigned NOT NULL AUTO_INCREMENT,
`source_name` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`source_live` BIT NOT NULL,
`source_link` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`source_type` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`source_area` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
UNIQUE (source_id)
);
#NL
CREATE TABLE IF NOT EXISTS `autobahn` (
`id` BIGINT unsigned NOT NULL AUTO_INCREMENT,
`name` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`source_id` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`loc` POINT NOT NULL,
`protocol_id` SMALLINT unsigned NOT NULL,
`link` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
UNIQUE (id)
);
#NL

188
apiHandler/autobahn/main.js Normal file
View File

@ -0,0 +1,188 @@
var fs = require("fs");
const bent = require("bent");
const mysql = require("mysql");
const tools = require("../../functions");
let con = undefined;
function initialize(connection) {
con = connection;
var array = fs
.readFileSync("./apiHandler/autobahn/db_table_config.sql")
.toString()
.split("\n");
for (i in array) {
con.query(array[i], function (err, result) {
tools.handleMysqlErrors(err, getModuleMeta().title);
console.log("Table created for autobahn");
});
}
const roadURL = "https://verkehr.autobahn.de/o/autobahn/";
const baseURL = "https://verkehr.autobahn.de/o/autobahn/#/services/webcam";
var hws = [];
var index_count_cams = 0;
console.log(
"[" +
getModuleMeta().title +
"] Starting requests"
);
const roadRequest = bent(roadURL, "GET", "json", 200);
roadRequest().then(
function (body) {
hws = body.roads;
for (h in hws) {
if (!hws[h].includes("/")) {
let camRequest = bent(
baseURL.replace("#", hws[h]),
"GET",
"json",
200
);
camRequest().then(
function (body) {
for (var i in body.webcam) {
var q =
"INSERT INTO autobahn (name, source_id, loc, protocol_id, link) VALUES (?, ?, POINT(?, ?), 0, ?)"
if (body.webcam[i].linkurl != "") {
index_count_cams += 1;
con.query(q, [body.webcam[i].title, getModuleMeta().exactName, body.webcam[i].coordinate.lat, body.webcam[i].coordinate.long, body.webcam[i].linkurl], function (err, result) {
tools.handleMysqlErrors(err, getModuleMeta().title);
});
}
}
},
function (err) {
console.warn(
"[" +
getModuleMeta().title +
"] Was unable to resolve request with error: " +
err
);
}
);
}
}
},
function (err) {
console.warn(
"[" +
getModuleMeta().title +
"] Was unable to resolve request with error: " +
err
);
}
);
console.log(
"[" +
getModuleMeta().title +
"] Done"
);
}
function queryData(boundNorthEastLatLng, boundSouthWestLatLng, minimal) {
/*
radius in km
*/
return new Promise(function (resolve, reject) {
// The Promise constructor should catch any errors thrown on
// this tick. Alternately, try/catch and reject(err) on catch.
var connection = con;
let elementsToQuery = "*";
if (minimal) {
elementsToQuery = "id,loc,source_id";
}
const sqlQuery = " \
SELECT " + elementsToQuery + " \
FROM autobahn \
WHERE MBRContains( \
GeomFromText( 'LINESTRING(? ?,? ?)' ), \
loc)";
connection.query(
sqlQuery,
[boundNorthEastLatLng.lat, boundNorthEastLatLng.lng, boundSouthWestLatLng.lat, boundSouthWestLatLng.lng],
function (err, rows, fields) {
// Call reject on error states,
// call resolve with results
if (err) {
return reject(err);
}
let results = [];
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e];
element = {
type: "webcam-iframe",
source: "Autobahn",
titel: currentRow.name,
description: "An example webcam with a url",
url: currentRow.link,
uid: currentRow.source_id + "_" + currentRow.id,
location: currentRow.loc,
icon: "cctvIcon",
};
if (minimal) {
const stripableElements = ["title", "description", "url", "type"];
for (h in stripableElements) {
delete element[stripableElements[h]];
}
}
results.push(element);
}
resolve(results);
}
);
});
}
function queryItem(uid) {
const uidParts = uid.split("_");
const sqlQuery = "SELECT * FROM " + getModuleMeta().exactName + " WHERE id=?";
return new Promise(function (resolve, reject) {
con.query(sqlQuery, [parseInt(uidParts[1])], function (err, rows, fields) {
if (err) {
return reject(err);
}
const results = [];
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e];
const element = {
type: "webcam-iframe",
source: "Autobahn",
titel: currentRow.name,
description: "An example webcam with a url",
url: currentRow.link,
uid: currentRow.source_id + "_" + currentRow.id,
location: currentRow.loc,
icon: "cctvIcon",
};
results.push(element);
}
resolve(results);
});
});
}
function getModuleMeta() {
return {
title: "Autobahn",
exactName: "autobahn",
country: ["DEU"],
tags: ["webcam"],
limited: false,
};
}
module.exports = { initialize, queryData, getModuleMeta, queryItem };

View File

@ -0,0 +1,2 @@
DROP TABLE IF EXISTS `customWebcams`
CREATE TABLE IF NOT EXISTS `customWebcams` ( `id` BIGINT unsigned NOT NULL AUTO_INCREMENT, `name` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `source_id` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `loc` POINT NOT NULL, `protocol_id` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `link` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, UNIQUE (id));

View File

@ -0,0 +1,12 @@
DROP TABLE IF EXISTS `customWebcams`
#NL
CREATE TABLE IF NOT EXISTS `customWebcams` (
`id` BIGINT unsigned NOT NULL AUTO_INCREMENT,
`name` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`source_id` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`loc` POINT NOT NULL,
`protocol_id` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`link` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
UNIQUE (id)
);
#NL

View File

@ -0,0 +1,145 @@
var fs = require("fs");
const bent = require("bent");
const mysql = require("mysql");
const tools = require("../../functions");
let con = undefined;
function initialize(connection) {
con = connection;
var array = fs
.readFileSync("./apiHandler/customWebcams/db_table_config.sql")
.toString()
.split("\n");
for (i in array) {
con.query(array[i], function (err, result) {
tools.handleMysqlErrors(err, getModuleMeta().title);
console.log("Table created for customWebcam");
});
}
console.log("[" + getModuleMeta().title + "] Starting requests");
const data = fs.readFileSync("./apiHandler/customWebcams/webcams.json");
const webcams = JSON.parse(data);
for (var i in webcams.cams) {
var q =
"INSERT INTO customWebcams (name, source_id, loc, protocol_id, link) VALUES (?, ?, POINT(?,?), ?, ?)"
con.query(q, [webcams.cams[i].title, getModuleMeta().exactName, webcams.cams[i].loc.lat, webcams.cams[i].loc.lng, webcams.cams[i].embedType, webcams.cams[i].url], function (err, result) {
tools.handleMysqlErrors(err, getModuleMeta().title);
});
}
console.log("[" + getModuleMeta().title + "] Done");
}
function queryData(boundNorthEastLatLng, boundSouthWestLatLng, minimal) {
return new Promise(function (resolve, reject) {
// The Promise constructor should catch any errors thrown on
// this tick. Alternately, try/catch and reject(err) on catch.
var connection = con;
let elementsToQuery = "*";
if (minimal) {
elementsToQuery = "id,loc,source_id";
}
const sqlQuery = " \
SELECT " + elementsToQuery + " \
FROM customWebcams \
WHERE MBRContains( \
GeomFromText( 'LINESTRING(? ?,? ?)' ), \
loc)";
con.query(sqlQuery, [boundNorthEastLatLng.lat, boundNorthEastLatLng.lng, boundSouthWestLatLng.lat, boundSouthWestLatLng.lng], function (err, rows, fields) {
// Call reject on error states,
// call resolve with results
if (err) {
return reject(err);
}
let results = [];
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e];
let ty = "";
if (currentRow.protocol_id == "IFRAME") {
ty = "webcam-iframe";
} else if (currentRow.protocol_id == "IMAGE") {
ty = "webcam-image";
}
element = {
type: ty,
source: "customWebcam",
titel: currentRow.name,
description: "An example webcam with a url",
url: currentRow.link,
uid: currentRow.source_id + "_" + currentRow.id,
location: currentRow.loc,
icon: "cctvIcon",
};
if (minimal) {
const stripableElements = ["title", "description", "url", "type"];
for (h in stripableElements) {
delete element[stripableElements[h]];
}
}
results.push(element);
}
resolve(results);
}
);
});
}
function queryItem(uid) {
const uidParts = uid.split("_");
const sqlQuery = "SELECT * FROM " + getModuleMeta().exactName + " WHERE id=?";
return new Promise(function (resolve, reject) {
con.query(sqlQuery, [parseInt(uidParts[1])], function (err, rows, fields) {
if (err) {
return reject(err);
}
const results = [];
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e];
let ty = "";
if (currentRow.protocol_id == "IFRAME") {
ty = "webcam-iframe";
} else if (currentRow.protocol_id == "IMAGE") {
ty = "webcam-image";
}
element = {
type: ty,
source: "customWebcam",
titel: currentRow.name,
description: "A web cam read from a file",
url: currentRow.link,
uid: currentRow.source_id + "_" + currentRow.id,
location: {
lat: currentRow.loc_lat,
lng: currentRow.loc_lng,
},
icon: "cctvIcon",
};
results.push(element);
}
resolve(results);
});
});
}
function getModuleMeta() {
return {
title: "custom webcams",
exactName: "customWebcams",
country: ["*"],
tags: ["webcam"],
limited: false,
};
}
module.exports = { initialize, queryData, getModuleMeta, queryItem };

View File

@ -0,0 +1,11 @@
{"cams": [
{
"title": "Blick auf den Großenbroder Südstrand",
"loc": {
"lat": 54.35780113081485,
"lng": 11.087519716842067
},
"url":"https://www.grossenbrode.de/webcamImage.php?secretmagickey=HEdqH9cwfqt2q35uuj75j5qdDdx3F7",
"embedType": "IMAGE"
}
]}

View File

114
apiHandler/example/main.js Normal file
View File

@ -0,0 +1,114 @@
const fs = require("fs");
const request = require("request");
const mysql = require("mysql");
const tools = require("../../functions");
let con = undefined;
function initialize(conection) {
console.error("!! THE EXAMPLE MODULE HAS BEEN ENABLED, NEVER ENABLE TO EXAMPLE MODULE !!")
con = conection;
const filePath = "./apiHandler/" + getModuleMeta().exactName + "/db_table_config.sql"
const array = fs
.readFileSync(filePath)
.toString()
.split("\n");
for (i in array) {
con.query(array[i], function (err, result) {
if (err) throw err;
console.log("Table created for " + getModuleMeta().title);
});
}
const requestURL = "https://example.com/api.json";
const mysqlQuery = "INSERT INTO " + getModuleMeta().exactName + " (" + getModuleMeta().exactName + "_id, " + getModuleMeta().exactName + "_name, " + getModuleMeta().exactName + "_description, source_id, " + getModuleMeta().exactName + "_loc_lat, " + getModuleMeta().exactName + "_loc_lng, " + getModuleMeta().exactName + "_protocol_id, " + getModuleMeta().exactName + "_link) VALUES (?, ?, ?, ?, ?, ?, ?, ?)";
request(requestURL, { json: true }, (err, res2, body) => {
if (err) {
throw err;
}
console.log("[" + getModuleMeta().title + "] Request done")
for (let c = 0; c < body.length; c++) {
const elm = body[c];
con.query(mysqlQuery, [c, elm.name, "", 0, elm.currentLocation.coordinates[1], elm.currentLocation.coordinates[0], 0, "https://api.opensensemap.org/boxes/" + elm._id], function (err, result) {
if (err) { throw (err) }
})
}
console.log("[" + getModuleMeta().title + "] Insert into Database done!")
});
}
function queryData(lat, lng, radius, minimal) {
/*
radius in km
*/
return new Promise(function (resolve, reject) {
// The Promise constructor should catch any errors thrown on
// this tick. Alternately, try/catch and reject(err) on catch.
let elementsToQuery = "*";
if (minimal) {
elementsToQuery = "id,loc_lat,loc_lng,source_id";
}
const sqlQuery =
"SELECT \
" +
elementsToQuery +
", \
( 6371 \
* acos( cos( radians(?) ) \
* cos( radians( " + getModuleMeta().exactName + "_loc_lat ) ) \
* cos( radians( " + getModuleMeta().exactName + "_loc_lng ) - radians(?) ) \
+ sin( radians(?) ) \
* sin( radians( " + getModuleMeta().exactName + "_loc_lat ) ) \
) \
) \
AS distance \
FROM " + getModuleMeta().exactName + " \
HAVING distance < ? \
ORDER BY distance";
con.query(sqlQuery, [lat, lng, lat, radius], function (err, rows, fields) {
// Call reject on error states,
// call resolve with results
if (err) {
return reject(err);
}
const results = [];
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e]
const element = {
"type": "request-info",
"source": getModuleMeta().title,
"titel": currentRow.example_name,
"description": "",
"url": currentRow.example_link,
"distance": currentRow.distance,
"location": {
"lat": currentRow.example_loc_lat,
"lng": currentRow.example_loc_lng
}
}
results.push(element)
}
resolve(results);
});
});
}
function getModuleMeta() {
return {
title: "Example",
exactName: "example",
country: ["*"],
limited: false,
};
}
module.exports = { initialize, queryData, getModuleMeta };

View File

@ -0,0 +1,2 @@
DROP TABLE IF EXISTS `freifunk`
CREATE TABLE IF NOT EXISTS `freifunk` ( `id` BIGINT unsigned NOT NULL AUTO_INCREMENT, `name` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL, `description` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL, `source_id` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL, `loc` POINT NOT NULL, `protocol_id` SMALLINT unsigned NOT NULL, `link` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL, UNIQUE (id));

View File

@ -0,0 +1,13 @@
DROP TABLE IF EXISTS `freifunk`
#NL
CREATE TABLE IF NOT EXISTS `freifunk` (
`id` BIGINT unsigned NOT NULL AUTO_INCREMENT,
`name` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`description` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`source_id` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`loc` POINT NOT NULL,
`protocol_id` SMALLINT unsigned NOT NULL,
`link` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
UNIQUE (id)
);
#NL

146
apiHandler/freifunk/main.js Normal file
View File

@ -0,0 +1,146 @@
const fs = require("fs");
const bent = require('bent')
const mysql = require("mysql");
const tools = require("../../functions");
let con = undefined;
function initialize(connection) {
con = connection;
const array = fs
.readFileSync("./apiHandler/freifunk/db_table_config.sql")
.toString()
.split("\n");
for (i in array) {
con.query(array[i], function (err, result) {
tools.handleMysqlErrors(err, getModuleMeta().title);
console.log("Table created for freifunk");
});
}
const freifunkDataUrl = "https://www.freifunk-karte.de/data.php";
const mysqlQuery =
"INSERT INTO freifunk (name, description, source_id, loc, protocol_id, link) VALUES (?, ?, ?, POINT(?, ?), ?, ?)";
const resulter = bent(freifunkDataUrl, 'GET', 'json', 200);
resulter().then(function(body){
console.log("[Freifunk] Request done");
for (let c = 0; c < body.allTheRouters.length; c++) {
const elm = body.allTheRouters[c];
con.query(
mysqlQuery,
[
elm.name,
elm.community,
getModuleMeta().exactName,
elm.lat,
elm.long,
0,
"https://www.freifunk-karte.de/",
],
function (err, result) {
tools.handleMysqlErrors(err, getModuleMeta().title);
}
);
}
console.log("[Freifunk] Insert into Database done!");
}, function(err){
console.warn("[" + getModuleMeta().title + "] Was unable to resolve request with error: " + err)
});
}
function queryData(boundNorthEastLatLng, boundSouthWestLatLng, minimal) {
return new Promise(function (resolve, reject) {
// The Promise constructor should catch any errors thrown on
// this tick. Alternately, try/catch and reject(err) on catch.
let elementsToQuery = "*";
if (minimal) {
elementsToQuery = "id,loc,source_id";
}
const sqlQuery = " \
SELECT " + elementsToQuery + " \
FROM freifunk \
WHERE MBRContains( \
GeomFromText( 'LINESTRING(? ?,? ?)' ), \
loc)";
con.query(sqlQuery, [boundNorthEastLatLng.lat, boundNorthEastLatLng.lng, boundSouthWestLatLng.lat, boundSouthWestLatLng.lng], function (err, rows, fields) {
// Call reject on error states,
// call resolve with results
if (err) {
return reject(err);
}
const results = [];
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e];
const element = {
type: "general-poi",
source: "Freifunk",
titel: currentRow.name,
description:
"This is an AP by the " +
currentRow.description +
" Freifunk Comunity",
url: currentRow.link,
uid: "freifunk_" + currentRow.id,
location: currentRow.loc,
icon: "wifiIcon"
};
if (minimal) {
const stripableElements = ["title", "description", "url", "type"];
for (h in stripableElements) {
delete element[stripableElements[h]];
}
}
results.push(element);
}
resolve(results);
});
});
}
function queryItem(uid){
const uidParts = uid.split("_");
const sqlQuery = "SELECT * FROM " + getModuleMeta().exactName + " WHERE id=?"
return new Promise(function (resolve, reject) {
con.query(sqlQuery, [parseInt(uidParts[1])], function (err, rows, fields) {
if (err) {
return reject(err);
}
const results = [];
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e];
const element = {
type: "general-poi",
source: "Freifunk",
titel: currentRow.name,
description:
"This is an AP by the " +
currentRow.description +
" Freifunk Comunity",
url: currentRow.link,
uid: "freifunk_" + currentRow.id,
location: currentRow.loc,
icon: "wifiIcon"
};
results.push(element);
}
resolve(results)
});
});
}
function getModuleMeta() {
return {
title: "Freifunk",
exactName: "freifunk",
country: ["*"],
tags: ["AP"],
limited: false,
};
}
module.exports = { initialize, queryData, getModuleMeta, queryItem };

View File

@ -0,0 +1,2 @@
DROP TABLE IF EXISTS `opensensemap`
CREATE TABLE IF NOT EXISTS `opensensemap` ( `id` BIGINT unsigned NOT NULL AUTO_INCREMENT, `name` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL, `description` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL, `source_id` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL, `loc` POINT NOT NULL, `protocol_id` SMALLINT unsigned NOT NULL, `link` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL, UNIQUE (id));

View File

@ -0,0 +1,13 @@
DROP TABLE IF EXISTS `opensensemap`
#NL
CREATE TABLE IF NOT EXISTS `opensensemap` (
`id` BIGINT unsigned NOT NULL AUTO_INCREMENT,
`name` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`description` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`source_id` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`loc` POINT NOT NULL,
`protocol_id` SMALLINT unsigned NOT NULL,
`link` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
UNIQUE (id)
);
#NL

View File

@ -0,0 +1,145 @@
const fs = require("fs");
const mysql = require("mysql");
const bent = require('bent')
const tools = require("../../functions");
let con = undefined;
function initialize(conection) {
con = conection;
const filePath =
"./apiHandler/" + getModuleMeta().exactName + "/db_table_config.sql";
const array = fs.readFileSync(filePath).toString().split("\n");
for (i in array) {
con.query(array[i], function (err, result) {
if (err) throw err;
console.log("Table created for " + getModuleMeta().title);
});
}
const requestURL = "https://api.opensensemap.org/boxes";
const mysqlQuery =
"INSERT INTO opensensemap (name, description, source_id, loc, protocol_id, link) VALUES (?, ?, ?, POINT(?, ?), ?, ?)";
const res = bent(requestURL, 'GET', 'json', 200);
res().then(function(response){
body = response
console.log("[" + getModuleMeta().title + "] Request done");
for (let c = 0; c < body.length; c++) {
const elm = body[c];
con.query(
mysqlQuery,
[
elm.name,
"",
getModuleMeta().exactName,
elm.currentLocation.coordinates[1],
elm.currentLocation.coordinates[0],
0,
"https://sensebox.github.io/opensensemap-widget/iframe.html?senseboxId=" + elm._id,
],
function (err, result) {
tools.handleMysqlErrors(err, getModuleMeta().title);
}
);
}
console.log("[" + getModuleMeta().title + "] Insert into Database done!");
}, function(err){
console.warn("[" + getModuleMeta().title + "] Was unable to resolve request with error: " + err)
});
}
function queryData(boundNorthEastLatLng, boundSouthWestLatLng, minimal) {
return new Promise(function (resolve, reject) {
// The Promise constructor should catch any errors thrown on
// this tick. Alternately, try/catch and reject(err) on catch.
let elementsToQuery = "*";
if (minimal) {
elementsToQuery = "id,loc,source_id";
}
// This is taken from https://stackoverflow.com/questions/21208697/select-all-geospatial-points-inside-a-bounding-box
const sqlQuery = " \
SELECT " + elementsToQuery + " \
FROM opensensemap \
WHERE MBRContains( \
GeomFromText( 'LINESTRING(? ?,? ?)' ), \
loc)";
con.query(sqlQuery, [boundNorthEastLatLng.lat, boundNorthEastLatLng.lng, boundSouthWestLatLng.lat, boundSouthWestLatLng.lng], function (err, rows, fields) {
// Call reject on error states,
// call resolve with results
if (err) {
return reject(err);
}
const results = [];
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e];
const element = {
type: "iframe",
source: getModuleMeta().exactName,
titel: currentRow.name,
description: "",
url: currentRow.link,
uid: "opensensemap_" + currentRow.id,
location: currentRow.loc,
icon: "temperatureIcon"
};
if (minimal) {
const stripableElements = ["title", "description", "url", "type"];
for (h in stripableElements) {
delete element[stripableElements[h]];
}
}
results.push(element);
}
resolve(results);
});
});
}
function queryItem(uid){
const uidParts = uid.split("_");
const sqlQuery = "SELECT * FROM `" + getModuleMeta().exactName + "` WHERE id=?"
return new Promise(function (resolve, reject) {
con.query(sqlQuery, [parseInt(uidParts[1])], function (err, rows, fields) {
if (err) {
return reject(err);
}
console.log()
const results = [];
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e];
const element = {
type: "iframe",
source: getModuleMeta().exactName,
titel: currentRow.name,
description: "",
url: currentRow.link,
uid: "opensensemap_" + currentRow.id,
distance: tools.roundOff(currentRow.distance, 4),
location: currentRow.loc,
icon: "temperatureIcon"
};
results.push(element);
}
resolve(results)
});
});
}
function getModuleMeta() {
return {
title: "OpenSensemap",
exactName: "opensensemap",
tags: ["temperature", "envoriment"],
country: ["*"],
limited: false,
};
}
module.exports = { initialize, queryData, getModuleMeta, queryItem };

View File

@ -0,0 +1,2 @@
DROP TABLE IF EXISTS `opentopia`
CREATE TABLE IF NOT EXISTS `opentopia` ( `id` BIGINT unsigned NOT NULL AUTO_INCREMENT, `name` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `source_id` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `loc` POINT NOT NULL, `protocol_id` SMALLINT unsigned NOT NULL, `link` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, UNIQUE (id), UNIQUE (`link`));

View File

@ -0,0 +1,13 @@
DROP TABLE IF EXISTS `opentopia`
#NL
CREATE TABLE IF NOT EXISTS `opentopia` (
`id` BIGINT unsigned NOT NULL AUTO_INCREMENT,
`name` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`source_id` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`loc` POINT NOT NULL,
`protocol_id` SMALLINT unsigned NOT NULL,
`link` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
UNIQUE (id),
UNIQUE (`link`)
);
#NL

View File

@ -0,0 +1,317 @@
var fs = require("fs");
const bent = require("bent");
const mysql = require("mysql");
const tools = require("../../functions");
const _ = require("underscore");
let con = undefined;
let config;
function requestWithPromise(n) {
const listURL =
"http://www.opentopia.com/hiddencam.php?showmode=standard&country=%2A&seewhat=highlyrated&p=";
return new Promise(function (resolve, reject) {
const request = bent(listURL + n, "GET", 200);
request().then(
function (body) {
body.text().then(function (text) {
resolve(text);
});
},
function (err) {
console.warn(
"[" +
getModuleMeta().title +
"] Was unable to resolve request with error: " +
err
);
}
);
});
}
function initialize(connection, configL, replaceOld = false) {
con = connection;
config = configL;
if(!fs.existsSync("./apiHandler/opentopia/FLAG")){
fs.writeFileSync("./apiHandler/opentopia/FLAG", "")
replaceOld = true
}
if (replaceOld) { // Just do it once, calling that API ain't cheap
var array = fs
.readFileSync("./apiHandler/opentopia/db_table_config.sql")
.toString()
.split("\n");
for (i in array) {
con.query(array[i], function (err, result) {
tools.handleMysqlErrors(err, getModuleMeta().title + "L32");
console.log("Table created for " + getModuleMeta().title);
});
}
let allProms = [];
for (let n = 0; n < 26; n++) {
allProms.push(requestWithPromise(n));
}
const mysqlQuery =
"INSERT INTO opentopia (name, source_id, loc, protocol_id, link) VALUES (?, ?, POINT(?, ?), ?, ?)";
Promise.all(allProms).then(function (res) {
for (l in res) {
const baseurl = "http://www.opentopia.com/webcam/";
let allResults = [];
// Process the pages
let part = res[l].split("<body>")[1];
part = part.split("/webcam/");
part.shift();
for (h in part) {
part[h] = part[h].replace("\t", "");
part[h] = part[h]
.split('target="_self">')[0]
.replace('"', "")
.split(">")[0];
}
pages = part;
camProms = [];
for (let pI = 0; pI < pages.length; pI++) {
camProms.push(
new Promise(function (resolve, reject) {
const url = baseurl + pages[pI] + "?viewmode=livevideo";
_.throttle(function () {
setTimeout(function () {
const request = bent(url, "GET", 200);
request().then(
function (body) {
body.text().then(function (text) {
let pre;
let result = {
name: "",
url: "",
location: { lat: 0, lng: 0 },
};
try {
const body = text;
let part = body.split("<body>")[1];
camPart = part.split('<div class="big">')[1];
camPart = camPart.split('<img src="')[1];
camPart = camPart.split('"')[0];
result.url = camPart;
part = part.split("Facility")[1];
part = part.split("Rating")[0];
part = part.split('<label class="right">')[1];
pre = part;
result.name = part.split("</label></div>")[0];
result.name = result.name
.replaceAll("\t", "")
.replaceAll(" ", "");
part = part.split('<label class="right geo">')[1];
part = part.split('<span class="latitude">')[1];
result.location.lat = parseFloat(
part.split("</span>")[0]
);
part = part.split('<span class="longitude">')[1];
result.location.lng = parseFloat(
part.split("</span>")[0]
);
resolve(result);
} catch (ex) {
if (pre == undefined) {
resolve([]);
} else {
let reqPart1;
let reqPart2;
try {
reqPart1 = pre
.split('locality">')[1]
.split("</label>")[0];
} catch (error) {
reqPart1 = pre
.split('region">')[1]
.split("</label>")[0];
}
reqPart2 = pre
.split('country-name">')[1]
.split("</label>")[0];
const geoCoderReq = bent(
"https://open.mapquestapi.com/geocoding/v1/address?key=" +
config.mapquest +
"&location=" +
reqPart1 +
"," +
reqPart2,
"GET",
"json",
200
);
geoCoderReq().then(function (body) {
result.location.lat =
body.results[0].locations[0].latLng.lat;
result.location.lng =
body.results[0].locations[0].latLng.lng;
resolve(result);
});
console.log(reqPart1 + "," + reqPart2);
}
}
});
},
function (err) {
console.warn(
"[" +
getModuleMeta().title +
"] Was unable to resolve request with error: " +
err
);
}
);
}, Math.floor(Math.random() * 500));
}, 200)();
})
);
let globalI = 0;
Promise.all(camProms).then(function (result) {
// console.log(result);
for (elm in result) {
const row = result[elm];
globalI++;
try {
con.query(
mysqlQuery,
[
row.name,
getModuleMeta().exactName,
row.location.lat,
row.location.lng,
0,
row.url,
],
function (err, result) {
if (err) {
if (err.code == "ER_DUP_ENTRY") {
//console.warn("[Opentopia] Skipped duplicate webcam");
} else {
tools.handleMysqlErrors(
err,
getModuleMeta().title + "L128"
);
}
}
}
);
} catch (error) {
/*console.warn(
"[Opentopia] Experienced an issue adding element to DB, error was: " +
error
);*/
}
}
});
}
}
});
}
}
function queryData(boundNorthEastLatLng, boundSouthWestLatLng, minimal) {
return new Promise(function (resolve, reject) {
// The Promise constructor should catch any errors thrown on
// this tick. Alternately, try/catch and reject(err) on catch.
var connection = con;
let elementsToQuery = "*";
if (minimal) {
elementsToQuery = "id,loc,source_id";
}
const sqlQuery = " \
SELECT " + elementsToQuery + " \
FROM opentopia \
WHERE MBRContains( \
GeomFromText( 'LINESTRING(? ?,? ?)' ), \
loc)";
con.query(sqlQuery, [boundNorthEastLatLng.lat, boundNorthEastLatLng.lng, boundSouthWestLatLng.lat, boundSouthWestLatLng.lng], function (err, rows, fields) {
// Call reject on error states,
// call resolve with results
if (err) {
tools.handleMysqlErrors(err, getModuleMeta().title + "-L187");
return reject(err);
}
let results = [];
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e];
element = {
type: "webcam-iframe",
source: "Opentopia",
titel: currentRow.name,
description: "An example webcam with a url",
url: currentRow.link,
uid: "opentopia_" + currentRow.id,
location: currentRow.loc,
icon: "cctvIcon",
};
if (minimal) {
const stripableElements = ["title", "description", "url", "type"];
for (h in stripableElements) {
delete element[stripableElements[h]];
}
}
results.push(element);
}
resolve(results);
}
);
});
}
function queryItem(uid) {
const uidParts = uid.split("_");
const sqlQuery = "SELECT * FROM " + getModuleMeta().exactName + " WHERE id=?";
return new Promise(function (resolve, reject) {
con.query(sqlQuery, [parseInt(uidParts[1])], function (err, rows, fields) {
if (err) {
tools.handleMysqlErrors(err, getModuleMeta().title + "L230");
return reject(err);
}
const results = [];
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e];
const element = {
type: "webcam-iframe",
source: "Opentopia",
titel: currentRow.name,
description: "An example webcam with a url",
url: currentRow.link,
uid: "opentopia_" + currentRow.id,
location: currentRow.loc,
icon: "cctvIcon",
};
results.push(element);
}
resolve(results);
});
});
}
function getModuleMeta() {
return {
title: "Opentopia",
exactName: "opentopia",
country: ["*"],
tags: ["webcam"],
limited: false,
};
}
module.exports = { initialize, queryData, getModuleMeta, queryItem };

View File

View File

@ -0,0 +1 @@

28
apiHandler/stub/main.js Normal file
View File

@ -0,0 +1,28 @@
let con = undefined;
function initialize(connection) {
con = connection;
// Do nothing and die.
}
function queryData(lat, lng, radius, minimal) {
// Does nothing.
return new Promise(function (resolve, reject) {resolve([]);});
}
function queryItem(uid) {
// Does nothing.
return new Promise(function (resolve, reject) {resolve([]);});
}
function getModuleMeta() {
return {
title: "Stub",
exactName: "stub",
country: [""],
tags: [""],
limited: true,
};
}
module.exports = { initialize, queryData, getModuleMeta, queryItem };

View File

@ -0,0 +1,2 @@
DROP TABLE IF EXISTS `ttnantenna`
CREATE TABLE IF NOT EXISTS `ttnantenna` ( `id` BIGINT unsigned NOT NULL AUTO_INCREMENT, `name` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL, `description` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL, `source_id` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL, `loc` POINT NOT NULL, `protocol_id` SMALLINT unsigned NOT NULL, `link` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL, UNIQUE (id));

View File

@ -0,0 +1,13 @@
DROP TABLE IF EXISTS `ttnantenna`
#NL
CREATE TABLE IF NOT EXISTS `ttnantenna` (
`id` BIGINT unsigned NOT NULL AUTO_INCREMENT,
`name` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`description` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`source_id` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`loc` POINT NOT NULL,
`protocol_id` SMALLINT unsigned NOT NULL,
`link` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
UNIQUE (id)
);
#NL

View File

@ -0,0 +1,175 @@
const fs = require("fs");
const bent = require("bent");
const mysql = require("mysql");
const tools = require("../../functions");
let con = undefined;
function initialize(connection) {
con = connection;
const array = fs
.readFileSync("./apiHandler/ttnantennas/db_table_config.sql")
.toString()
.split("\n");
for (i in array) {
con.query(array[i], function (err, result) {
if (err) throw err;
console.log("Table created for ttnantenna");
});
}
const ttnGateways =
"https://www.thethingsnetwork.org/gateway-data/";
const mysqlQuery =
"INSERT INTO ttnantenna (name, description, source_id, loc, protocol_id, link) VALUES (?, ?, ?, POINT(?, ?), ?, ?)";
const ttnGateRequest = bent(ttnGateways, "GET", "json", 200);
ttnGateRequest().then(function(body){
console.log("[TheThingsnetwork Gateways] Request done");
let i = 0;
for (const key in body) {
const elm = body[key];
if (elm.location != undefined) {
if(elm.attributes == undefined){
elm.attributes = {frequency_plan: "Unknown", placement: "Unknown"}
}else{
if(elm.attributes.frequency_plan == undefined){
elm.attributes.frequency_plan = "Unknown"
}
if(elm.attributes.placement == undefined){
elm.attributes.placement = "Unknown"
}
}
if (
elm.location.latitude != undefined &&
elm.location.longitude != undefined
) {
con.query(
mysqlQuery,
[
key,
elm.description +
"<br>Frequency plan: " +
elm.attributes.frequency_plan +
"<br> Altitude: " +
elm.location.altitude
+ "<br>Placement: " + elm.attributes.placement,
getModuleMeta().exactName,
elm.location.latitude,
elm.location.longitude,
0,
"",
],
function (err, result) {
tools.handleMysqlErrors(err, getModuleMeta().title);
}
);
i++
}
}
}
console.log("[TheThingsnetwork Gateways] Insert into Database done!");
},
function (err) {
console.warn(
"[" +
getModuleMeta().title +
"] Was unable to resolve request with error: " +
err
);
});
}
function queryData(boundNorthEastLatLng, boundSouthWestLatLng, minimal) {
return new Promise(function (resolve, reject) {
// The Promise constructor should catch any errors thrown on
// this tick. Alternately, try/catch and reject(err) on catch.
var connection = con;
let elementsToQuery = "*";
if (minimal) {
elementsToQuery = "id,loc,source_id";
}
const sqlQuery = " \
SELECT " + elementsToQuery + " \
FROM ttnantenna \
WHERE MBRContains( \
GeomFromText( 'LINESTRING(? ?,? ?)' ), \
loc)";
con.query(sqlQuery, [boundNorthEastLatLng.lat, boundNorthEastLatLng.lng, boundSouthWestLatLng.lat, boundSouthWestLatLng.lng], function (err, rows, fields) {
// Call reject on error states,
// call resolve with results
if (err) {
return reject(err);
}
const results = [];
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e];
const element = {
type: "general-poi",
source: "ttnantennas",
titel: currentRow.name,
description: currentRow.description,
url: currentRow.link,
uid: "ttnantennas_" + currentRow.id,
location: currentRow.loc,
icon: "signalIcon"
};
if (minimal) {
const stripableElements = ["title", "description", "url", "type"];
for (h in stripableElements) {
delete element[stripableElements[h]];
}
}
results.push(element);
}
resolve(results);
});
});
}
function queryItem(uid) {
const uidParts = uid.split("_");
const sqlQuery = "SELECT * FROM ttnantenna WHERE id=?";
return new Promise(function (resolve, reject) {
con.query(sqlQuery, [parseInt(uidParts[1])], function (err, rows, fields) {
if (err) {
return reject(err);
}
const results = [];
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e];
const element = {
type: "general-poi",
source: "ttnantennas",
titel: currentRow.name,
description: currentRow.description,
url: currentRow.link,
uid: "ttnantennas_" + currentRow.id,
location: {
lat: currentRow.loc_lat,
lng: currentRow.loc_lng,
},
};
results.push(element);
}
resolve(results);
});
});
}
function getModuleMeta() {
return {
title: "TheThingsnetwork Gateways",
exactName: "ttnantennas",
country: ["*"],
limited: false,
};
}
module.exports = { initialize, queryData, getModuleMeta, queryItem };

View File

@ -0,0 +1,2 @@
DROP TABLE IF EXISTS `ttncomms`
CREATE TABLE IF NOT EXISTS `ttncomms` ( `id` BIGINT unsigned NOT NULL AUTO_INCREMENT, `name` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `description` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `source_id` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `loc` POINT NOT NULL, `protocol_id` SMALLINT unsigned NOT NULL, `link` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, UNIQUE (id));

View File

@ -0,0 +1,13 @@
DROP TABLE IF EXISTS `ttncomms`
#NL
CREATE TABLE IF NOT EXISTS `ttncomms` (
`id` BIGINT unsigned NOT NULL AUTO_INCREMENT,
`name` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`description` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`source_id` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`loc` POINT NOT NULL,
`protocol_id` SMALLINT unsigned NOT NULL,
`link` TEXT CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
UNIQUE (id)
);
#NL

165
apiHandler/ttncomms/main.js Normal file
View File

@ -0,0 +1,165 @@
const fs = require("fs");
const bent = require('bent')
const tools = require("../../functions");
let con = undefined;
function initialize(connection) {
con = connection;
const array = fs
.readFileSync("./apiHandler/ttncomms/db_table_config.sql")
.toString()
.split("\n");
for (i in array) {
con.query(array[i], function (err, result) {
if (err) throw err;
console.log("Table created for ttncomms");
});
}
const ttnCommUrl =
"https://www.thethingsnetwork.org/community.json";
const mysqlQuery =
"INSERT INTO ttncomms (name, description, source_id, loc, protocol_id, link) VALUES (?, ?, ?, POINT(?, ?), ?, ?)";
const res = bent(ttnCommUrl, 'GET', 'json', 200);
res().then(function(body){
console.log("[TheThingsnetwork Communities] Request done");
for (let c = 0; c < body.length; c++) {
const elm = body[c];
if (elm.lat != null && elm.lon != null) {
con.query(
mysqlQuery,
[
"TTN Community " + elm.title,
"",
getModuleMeta().exactName,
elm.lat,
elm.lon,
0,
"https://www.thethingsnetwork.org" + elm.path,
],
function (err, result) {
if (err) {
throw err;
}
}
);
}
}
console.log("[TheThingsnetwork Communities] Insert into Database done!");
}, function(err){
console.warn("[" + getModuleMeta().title + "] Was unable to resolve request with error: " + err)
});
}
function queryData(boundNorthEastLatLng, boundSouthWestLatLng, minimal) {
// Create a new promise to return, DBs are slow.
return new Promise(function (resolve, reject) {
// The Promise constructor should catch any errors thrown on
// this tick. Alternately, try/catch and reject(err) on catch.
let elementsToQuery = "*";
if (minimal) {
elementsToQuery = "id,loc,source_id";
}
// This is taken from https://stackoverflow.com/questions/21208697/select-all-geospatial-points-inside-a-bounding-box
const sqlQuery = " \
SELECT " + elementsToQuery + " \
FROM airspy \
WHERE MBRContains( \
GeomFromText( 'LINESTRING(? ?,? ?)' ), \
loc)";
con.query(sqlQuery, [boundNorthEastLatLng.lat, boundNorthEastLatLng.lng, boundSouthWestLatLng.lat, boundSouthWestLatLng.lng], function (err, rows, fields) {
// Call reject on error states,
// call resolve with results
if (err) {
return reject(err);
}
const results = [];
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e];
const element = {
type: "general-poi",
source: "ttncomms",
titel: currentRow.name,
description: "This is a TheThingsnetwork Comunity. URL: " + currentRow.link,
url: currentRow.link,
uid: "ttncomms_" + currentRow.id,
location: currentRow.loc,
};
if (minimal) {
const stripableElements = ["title", "description", "url", "type"];
for (h in stripableElements) {
delete element[stripableElements[h]];
}
}
results.push(element);
}
resolve(results);
});
});
}
function queryItem(uid){
const uidParts = uid.split("_");
console.log("[ttncomms] Query item with uid: " + uid + " using id: " + parseInt(uidParts[1]));
const sqlQuery = "SELECT * FROM " + getModuleMeta().exactName + " WHERE id=?"
return new Promise(function (resolve, reject) {
con.query(sqlQuery, [parseInt(uidParts[1])], function (err, rows, fields) {
if (err) {
return reject(err);
}
console.log("[ttncomms] Query done, result: " + rows);
let results = [];
if(rows.length > 0){
for (let e = 0; e < rows.length; e++) {
const currentRow = rows[e];
const element = {
type: "general-poi",
source: "ttncomms",
titel: currentRow.name,
description: "This is a TheThingsnetwork Comunity. URL: " + currentRow.link,
url: currentRow.link,
uid: "ttncomms_" + currentRow.id,
location: {
lat: currentRow.loc_lat,
lng: currentRow.loc_lng,
},
};
results.push(element);
}
}/*else{
results = [{
type: "general-poi",
source: "ttncomms",
titel: "Invalid",
description: "This is broken. Something is broken.",
url: "INVALID",
uid: uid,
location: {
lat: 0,
lng: 0,
},
}];
}*/
resolve(results)
});
});
}
function getModuleMeta() {
return {
title: "TheThingsnetwork Communities",
exactName: "ttncomms",
country: ["*"],
limited: false,
};
}
module.exports = { initialize, queryData, getModuleMeta, queryItem };

15
config/default.json.save Normal file
View File

@ -0,0 +1,15 @@
{
"fontAwesome": "d7b80a780b",
"mapboxAccessToken": "pk.eyJ1IjoiZ3JleWRpYW1vbmQiLCJhIjoiY2p5NXJyZmZtMDlzYjNibGV2cnZ1MTZxbyJ9.7kB9q9Hijmx9MewGM0F0MQ",
"cookieSecret": "fdgdfgdgsfg156516ds561156dsf10g65d1f0g651fd651/",
"database": {
"host": "localhost",
"user": "knowMap",
"password": "uwCG5rVc51qv4Tb8",
"database": "knowMapexit",
"multipleStatements": true,
"charset": "utf8mb4"
},
"env": "DEV"
}

View File

@ -0,0 +1,16 @@
{
"fontAwesome": "",
"mapboxAccessToken": "",
"cookieSecret": "",
"mapquest": "",
"database": {
"host": "localhost",
"user": "",
"password": "",
"database": ""
},
"env": "DEV",
"maint": false,
"betaMode": true,
"port": 3000
}

25
functions.js Normal file
View File

@ -0,0 +1,25 @@
// Rounds a number to ˋplacesˋ amount of digits
// Example roundOff(5.2434234234, 3) => 5.243
let roundOff = (num, places) => {
const x = Math.pow(10, places);
return Math.round(num * x) / x;
};
function handleMysqlErrors(err, title) {
if (err) {
if (err.code == "ECONNREFUSED") {
console.log(
"⚠ Connection to database failed. Is the database server running? ⚠"
);
process.exit(5);
} else if (err.code == "ECONNRESET") {
console.log("⚠ Module " + title + " experienced a connection reset ⚠");
process.exit(5);
} else {
console.warn("!!!!!!!!!!!!!!!! NOW THROWING " + err.code + " !!!!!!!!!!!!!!!!")
throw err;
}
}
}
module.exports = { roundOff, handleMysqlErrors };

10
libs/jsonHand.lib.js Normal file
View File

@ -0,0 +1,10 @@
function IsJsonString(str) {
try {
JSON.parse(str);
} catch (e) {
return false;
}
return true;
}
module.exports = { IsJsonString }

26
libs/metaHandler.lib.js Normal file
View File

@ -0,0 +1,26 @@
function returnTags(module){
const meta = module.getModuleMeta()
if(meta.features != undefined){
if(meta.features.includes("tags")){
return(meta.tags)
}
}else if(meta.tags != undefined){
return(meta.tags)
}else{
return([]) // Assume the module does not support tags
}
}
function returnCountries(module){
const meta = module.getModuleMeta()
if(meta.features != undefined){
if(meta.features.includes("countries")){
return(meta.country)
}
}if(meta.country != undefined){
return(meta.country)
}else{
return(["*"]) // Assume the module does not support tags
}
}
module.exports = { returnTags, returnCountries }

View File

@ -0,0 +1,10 @@
{
"fields": [
"name",
"licenseType",
"link",
"comment",
"installedVersion",
"author"
]
}

370
main.ts Normal file
View File

@ -0,0 +1,370 @@
/***
* Copyright © 2021 Sören W. Oesterwind (TheGreydiamond). All rights reserved.
*/
// Requires (Imports)
const express = require("express");
const fs = require("fs");
const mysql = require('mysql');
const autobahnAPI = require("./apiHandler/autobahn/main.js");
/*const ffhAPI = require("./apiHandler/ffh/main.js");
const stadtkoelnAPI = require("./apiHandler/ffh/main.js");*/
const customWebcamAPI = require("./apiHandler/customWebcams/main.js");
const airspyAPI = require("./apiHandler/airspydirectory/main");
const ttncommsAPI = require("./apiHandler/ttncomms/main");
const freifunkAPI = require("./apiHandler/freifunk/main");
const senseboxAPI = require("./apiHandler/opensensemap/main.js");
const opentopiaAPI = require("./apiHandler/opentopia/main.js");
const ttnGateway = require("./apiHandler/ttnantennas/main");
const stubMod = require("./apiHandler/stub/main");
const _ = require("underscore");
const chalk = require('chalk');
const func = require("./functions.js")
const minify = require('express-minify');
const compression = require('compression');
const cookieParser = require('cookie-parser')
const bodyParser = require('body-parser');
const Sentry = require('@sentry/node');
const Tracing = require('@sentry/tracing');
const bent = require("bent");
const Influx = require("influx");
function padLeadingZeros(num, size) {
let s = num + "";
while (s.length < size) s = "0" + s;
return s;
}
const enabledModules = [airspyAPI, autobahnAPI, customWebcamAPI, freifunkAPI, senseboxAPI, opentopiaAPI, ttnGateway, ttncommsAPI];
// Webserver init
const app = express();
const allTags = {};
const metaGlobals = { "desc": "Pointsight is a centralized geolocalized API aggragator.", "titlePrefx": "Pointsight - " }
// Skeleton Variables
let jsonConfigGlobal = {
fontAwesome: undefined,
mapboxAccessToken: undefined,
cookieSecret: undefined,
mapquest: undefined,
here: undefined,
sentryDsn: undefined,
database: {
host: "127.0.0.1",
user: "pointsight",
password: "",
database: "pointsight",
customSettings: {
wait_timout: 28800
}
},
env: "PROD",
maint: true,
betaMode: false,
port: 3000,
adress: '127.0.0.1',
taxonomyCacheInterval: 5,
metrics: {
influx: {
enable: false,
host: "127.0.0.1",
database: "pointsight",
user: "pointsight",
password: ""
},
writeMetricsToMySQL: true
}
}
// Load config
try {
const data = fs.readFileSync("config/default.json", "utf8");
jsonConfigGlobal = _.extend(jsonConfigGlobal, JSON.parse(data));
} catch (error) {
console.error(
"While reading the config an error occured. The error was: " + error
);
}
console.log(jsonConfigGlobal.metrics.influx)
Sentry.init({
dsn: jsonConfigGlobal.sentryDsn,
integrations: [
// enable HTTP calls tracing
new Sentry.Integrations.Http({ tracing: true }),
// enable Express.js middleware tracing
new Tracing.Integrations.Express({ app }),
new Tracing.Integrations.Mysql({ useMysql: true })
],
// Set tracesSampleRate to 1.0 to capture 100%
// of transactions for performance monitoring.
// We recommend adjusting this value in production
tracesSampleRate: 0.8,
});
app.use(Sentry.Handlers.requestHandler());
app.use(Sentry.Handlers.tracingHandler());
const con = mysql.createConnection(jsonConfigGlobal.database);
const connectionPool = mysql.createPool(jsonConfigGlobal.database);
const allPoolConnections = [];
let influxD = undefined;
let isInfluxReady = false;
let lastHttpRequest = Math.floor(Date.now() / 1000)
if(jsonConfigGlobal.metrics.influx.enable){
console.log("[Influx] Connecting to InfluxDB...");
influxD = new Influx.InfluxDB({
host: jsonConfigGlobal.metrics.influx.host,
database: jsonConfigGlobal.metrics.influx.database,
user: jsonConfigGlobal.metrics.influx.user,
password: jsonConfigGlobal.metrics.influx.password,
schema: [
{
measurement: 'apitaxonomy',
fields: { calls: Influx.FieldType.INTEGER },
tags: []
}
]
});
console.log("[Influx] Checking if database exists...");
influxD.getDatabaseNames()
.then(names => {
if (!names.includes('pointsight')) {
return influxD.createDatabase('pointsight');
}
})
.then(() => {
console.log("[Influx] Database ready.");
isInfluxReady = true;
})
.catch(error => console.log({ error }));
}
connectionPool.getConnection((err, conn) => {
conn.on('error', function (err) {
console.log("----[MYSQL ERROR]----")
console.log(err); // 'ER_BAD_DB_ERROR'
console.log("[MYSQL ERROR] Connection pool error");
console.log("----]MYSQL ERROR[----")
});
func.handleMysqlErrors(err)
//Create system tabels
const queries = ["CREATE TABLE IF NOT EXISTS `apikeys` ( `id` BIGINT NOT NULL AUTO_INCREMENT , `apikey` VARCHAR(64) NOT NULL , `owner` VARCHAR(64) NOT NULL , `hosts` JSON NOT NULL , `created` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP , `expire` DATETIME NOT NULL , PRIMARY KEY (`id`));", "CREATE TABLE IF NOT EXISTS `betatokens` ( `id` BIGINT NOT NULL AUTO_INCREMENT , `token` VARCHAR(64) NOT NULL , `owner` VARCHAR(64) NOT NULL, `created` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP , `expire` DATETIME NOT NULL , PRIMARY KEY (`id`));",
"CREATE TABLE IF NOT EXISTS `reports` ( `id` INT NOT NULL AUTO_INCREMENT , `point_id` VARCHAR(512) NOT NULL , `mail` VARCHAR(512) NOT NULL , `comment` VARCHAR(2048) NOT NULL , PRIMARY KEY (`id`)) ENGINE = InnoDB;",
"CREATE TABLE IF NOT EXISTS `apitaxonomy` ( `id` INT NOT NULL AUTO_INCREMENT , `apiKey` VARCHAR(255) NOT NULL , `date` DATE NOT NULL DEFAULT CURRENT_TIMESTAMP , `calls` INT NOT NULL DEFAULT '0' , PRIMARY KEY (`id`)) ENGINE = InnoDB;"]
queries.forEach(element => {
conn.query(element, function (err) {
if (err) throw err;
console.log("Table created for Main system");
});
});
conn.release()
})
for (let i = 0; i < enabledModules.length; i++) {
connectionPool.getConnection((err, conn) => {
if (err) {
if (err.code == "ECONNREFUSED") {
console.log(chalk.red("⚠ Connection to database failed. Is the database server running? ⚠"))
server.close()
process.exit(5)
} else {
throw err;
}
}
allPoolConnections.push(conn);
conn.on('error', function (err) {
console.log("----[MYSQL ERROR]----")
console.log("[MYSQL ERROR] Module connection error");
console.log(err); // 'ER_BAD_DB_ERROR'
if (err.code == "ECONNRESET") {
connectionPool.getConnection((err, connP) => {
if (err) throw err;
conn = connP // Trying to mitigate ECONNRESET issues, even tho this didn't fix it lol
})
}
console.log("----]MYSQL ERROR[----")
});
_.each(allPoolConnections, function (arg) { setInterval(function () { arg.query("SELECT 1") }, jsonConfigGlobal.database.customSettings.wait_timout - 50) })
try {
enabledModules[i].initialize(conn, jsonConfigGlobal)
const currTags = enabledModules[i].getModuleMeta().tags
for (const tagy in currTags) {
const tag = currTags[tagy]
if (_.isFinite(allTags[tag])) {
allTags[tag] = allTags[tag] + 1;
} else {
allTags[tag] = 1;
}
}
} catch (error) {
console.warn(chalk.red("⚠ Module " + enabledModules[i].getModuleMeta().title + " failed to initialize() ⚠" + "\n Error: " + error + "\n Stacktrace: " + error.stack))
enabledModules[i] = stubMod;
}
});
setInterval(function () {
for (const conyT in allPoolConnections) {
allPoolConnections[conyT].query("SELECT 1;", function (err) {
if (err) throw err;
console.log("Querrying heartbeat for connection pool number: " + conyT);
});
}
}, jsonConfigGlobal.database.customSettings.wait_timout * 1000)
}
app.use(express.static("static"));
// A crappy logging middleware, please don't look at this
app.use(function (req, res, next) {
lastHttpRequest = Math.floor(Date.now() / 1000)
const date_ob = new Date();
const date = ("0" + date_ob.getDate()).slice(-2);
// current month
const month = ("0" + (date_ob.getMonth() + 1)).slice(-2);
// current year
const year = date_ob.getFullYear();
// current hours
const hours = padLeadingZeros(date_ob.getHours(), 2)
// current minutes
const minutes = padLeadingZeros(date_ob.getMinutes(), 2)
// current seconds
const seconds = padLeadingZeros(date_ob.getSeconds(), 2)
// prints date & time in YYYY-MM-DD HH:MM:SS format
console.log("[" + year + "-" + month + "-" + date + " " + hours + ":" + minutes + ":" + seconds + "] " + req.method + " " + req.path);
next();
});
app.use(bodyParser.urlencoded({ extended: true }));
app.use(cookieParser(jsonConfigGlobal.cookieSecret));
if (jsonConfigGlobal.betaMode) {
require('./middleware/beta.middleware')(app, con, jsonConfigGlobal);
}
// API Taxonomy
const apiTaxonomyCache = {};
function flushApiTaxonomyCache():void {
if(jsonConfigGlobal.metrics.writeMetricsToMySQL){
const updateSql = "UPDATE apitaxonomy SET calls = calls+? WHERE apikey LIKE ? AND DATE(date) = CURDATE()";
const insertSql = "INSERT INTO apitaxonomy (apikey, calls) VALUES(?, ?);"
console.log( Object.entries(apiTaxonomyCache).length + " entries in apiTaxonomyCache")
for (const [key, value] of Object.entries(apiTaxonomyCache)) {
con.query(updateSql, [value, key], function (err, result) {
if (err) {
console.error(err);
}
if(!jsonConfigGlobal.metrics.influx.enable){
apiTaxonomyCache[key] = 0;
}
if (result.affectedRows == 0) {
con.query(insertSql, [key, value], function (err, result) {
if (err) {
console.error(err);
}
});
}
});
}
}else{
if(!jsonConfigGlobal.metrics.influx.enable){
console.log(chalk.red("⚠ MySQL Metrics writing has been disabled, but Influx is not setup! ⚠"));}
}
if(jsonConfigGlobal.metrics.influx.enable){
const toWritePoints = [];
console.log( Object.entries(apiTaxonomyCache).length + " entries in apiTaxonomyCache in Influx")
for (const [key, value] of Object.entries(apiTaxonomyCache)) {
const val = apiTaxonomyCache[key]
toWritePoints.push({
measurement: 'apitaxonomy',
tags: {},
fields: { calls: val },
timestamp: new Date,
})
apiTaxonomyCache[key] = 0;
}
influxD.writePoints(toWritePoints, {
database: jsonConfigGlobal.metrics.influx.database,
precision: 's',
})
.catch(error => {
console.error(`[InfluxDB] Error saving data to InfluxDB! ${error.stack}`)
});
}
}
if(jsonConfigGlobal.env == "DEV"){
jsonConfigGlobal.taxonomyCacheInterval = 0.5;
}
setInterval(function () {
flushApiTaxonomyCache()
console.log("[API TAXONOMY] Wrote taxonomy to database");
}, jsonConfigGlobal.taxonomyCacheInterval * 60 * 1000); // Every `taxonomyCacheInterval` minutes
connectionPool.getConnection((err, conn) => {
conn.on('error', function (err) {
console.log("----[MYSQL ERROR]----")
console.log("[MYSQL ERROR] System connection error");
console.log(err);
console.log("----]MYSQL ERROR[----")
});
setInterval(function () {
conn.query("SELECT 1;", function (err) {
if (err) throw err;
console.log("Querrying heartbeat for system connection");
});
}, jsonConfigGlobal.database.customSettings.wait_timout * 1000) // Every 20 Minutes
func.handleMysqlErrors(err)
require('./middleware/maint.middleware')(app, jsonConfigGlobal, metaGlobals);
require('./middleware/apitoken.middleware')(app, con, apiTaxonomyCache);
require('./routes/api.route.ts')(app, enabledModules, conn, { tags: allTags });
require('./routes/index.route.js')(app, metaGlobals, jsonConfigGlobal.fontAwesome, jsonConfigGlobal.mapboxAccessToken, jsonConfigGlobal);
app.use(Sentry.Handlers.errorHandler());
require('./routes/error.route.js')(app, jsonConfigGlobal, metaGlobals); // Make sure this is always last
app.use(minify());
app.use(compression());
})
let server = undefined;
setInterval(function () {
if (Math.floor(Date.now() / 1000) - lastHttpRequest >= 60 * 60) { // One hour after the last http request
const request = bent("https://pointsight.project-name-here.de/api/getPOI?boundingEast=55.21649013168979;10.696563720703127&boundingWest=54.59354980818523;9.825897216796877&key=b03f8aaf-1f32-4d9e-914a-9a50f904833d", "GET", "json", 200);
request().then(function (body) {
console.log("Tried to pull POI from pointsight.project-name-here.de")
})
}
}, 60000);
process.on('SIGINT', function () {
console.log("Caught interrupt signal and shutting down gracefully");
flushApiTaxonomyCache(); // Save the api taxonomy cache before quitting
server.close(); // Make th express server stop
_.each(allPoolConnections, function (arg) { arg.release(); }) // Let go off all mysql pool connections
con.end(); // Close the system's mysql connection
Sentry.close(2000).then(function () { // Wait for sentry to quit
console.log(chalk.blue("👋 Goodbye! 👋"))
process.exit(); // Quit the application
});
});
// Start server
server = app.listen(jsonConfigGlobal.port, jsonConfigGlobal.adress, () => {
console.log(`The Pointsight is running at http://localhost:${jsonConfigGlobal.port}`)
setTimeout(function () {
if (jsonConfigGlobal.env == "DEV") {
console.log(allTags);
}
}, 1000)
})

View File

@ -0,0 +1,86 @@
module.exports = function (app, con, apiTaxonomyCache) {
const _ = require("underscore");
function isFutureDate(value) {
const d_now = new Date();
const d_inp = new Date(value);
return d_now.getTime() <= d_inp.getTime();
}
app.use(function (req, res, next) {
// API key handling middleware
if (req.path.includes("/api/")) {
if (req.query.key != undefined) {
const sql = "SELECT * FROM apikeys WHERE apikey LIKE ? LIMIT 1";
con.query(sql, [req.query.key], function (err, result) {
if (err) {
throw err;
}
if (result.length == 0) {
// There is atleast one result
res.status(401);
res.setHeader("Content-Type", "application/json");
res.send(
JSON.stringify({ state: "Failed", message: "Invalid API key" })
);
} else {
// console.log(req.headers);
if (
JSON.parse(result[0].hosts).includes(req.hostname) ||
JSON.parse(result[0].hosts).includes("*")
) {
// Is the key even allowed for this host?
if (isFutureDate(result[0].expire)) { // Has the key expired?
next(); // Allow it to pass
if(!_.isFinite(apiTaxonomyCache[req.query.key])){
apiTaxonomyCache[req.query.key] = 0;
}
apiTaxonomyCache[req.query.key] = apiTaxonomyCache[req.query.key]+1;
// console.log(apiTaxonomyCache)
/*const updateSql = "UPDATE apitaxonomy SET calls = calls+1 WHERE apikey LIKE ? AND DATE(date) = CURDATE()";
const insertSql = "INSERT INTO apitaxonomy (apikey, calls) VALUES(?, ?);"
con.query(updateSql, [req.query.key], function (err, result) {
if (err) {
console.error(err);
}
if(result.affectedRows == 0) {
con.query(insertSql, [req.query.key, 1], function (err, result) {
if (err) {
console.error(err);
}
});
}
});*/
} else {
// Yes? Then no passing!
res.status(401);
res.setHeader("Content-Type", "application/json");
res.send(
JSON.stringify({
state: "Failed",
message: "Expired API key",
})
);
}
} else {
res.status(401);
res.setHeader("Content-Type", "application/json");
res.send(
JSON.stringify({
state: "Failed",
message: "Invalid Hostname for API key",
})
);
}
}
});
} else {
res.status(401);
res.setHeader("Content-Type", "application/json");
res.send(
JSON.stringify({ state: "Failed", message: "Missing API key" })
);
}
} else {
next();
}
});
};

View File

@ -0,0 +1,142 @@
module.exports = function (app, con, config) {
const uuid = require("uuid");
const Eta = require("eta");
const fs = require("fs");
function isFutureDate(value) {
const d_now = new Date();
const d_inp = new Date(value);
return d_now.getTime() <= d_inp.getTime();
}
app.use(function (req, res, next) {
res.locals.valid = true;
if (
req.path.includes("/api/") ||
req.path.includes("favicon") ||
req.path.includes("/betaLogin") ||
req.path.includes("/redirectUrl") ||
req.path.includes("/beta/Invite")
) {
next();
} else {
if (config.env == "DEV") {
console.log("[beta.middleware.js]", req.path);
}
if (uuid.validate(req.signedCookies.betaToken)) {
next();
} else {
// res.cookie('betaToken', uuid.v4(), { signed: true })
const data = fs.readFileSync("templates/redr.eta.html", "utf8");
res.locals.valid = false;
res.send(
Eta.render(data, {
siteTitel: "Pointsight - BetaLogin",
redirectUrl: "/betaLogin",
})
);
}
}
});
app.post("/betaLogin", function (req, res) {
const sql = "SELECT * FROM betatokens WHERE token LIKE ? LIMIT 1";
con.query(sql, [req.body.betatoken], function (err, result) {
if (err) {
throw err;
}
if (result.length == 0) {
// There is atleast one result
res.status(401);
res.setHeader("Content-Type", "application/json");
res.send(
JSON.stringify({ state: "Failed", message: "Invalid API key" })
);
} else {
if (isFutureDate(result[0].expire)) {
// Has the key expired?
res.status(200);
res.cookie("betaToken", uuid.v4(), { signed: true });
res.redirect("/");
} else {
// Yes? Then no passing!
res.status(401);
res.setHeader("Content-Type", "application/json");
res.send(
JSON.stringify({ state: "Failed", message: "Expires API key" })
);
}
}
});
console.log(req.body);
});
app.get("/beta/Invite", function (req, res) {
const myInv = req.query.invite; // Invite code
let invtee = req.query.invitee; // Used for personalized invites
if(invtee == undefined){
invtee = "Someone"; // Fallback if none is given
}
if (myInv == undefined) {
const data = fs.readFileSync("templates/redr.eta.html", "utf8");
res.locals.valid = false;
res.send(
Eta.render(data, {
siteTitel: "Pointsight",
redirectUrl: "/",
})
);
} else {
const data = fs.readFileSync("templates/beta/invitePage.eta.html", "utf8");
res.send(
Eta.render(data, {
siteTitel: "Pointsight - BetaToken - Invite",
invite: myInv,
invitee: invtee
})
);
}
});
app.get("/betaLogin", function (req, res) {
if (req.query.betaKey != undefined) {
console.log(req.query.betaKey);
const sql = "SELECT * FROM betatokens WHERE token LIKE ? LIMIT 1";
con.query(sql, [req.query.betaKey], function (err, result) {
if (err) {
throw err;
}
if (result.length == 0) {
// There is atleast one result
res.status(401);
res.setHeader("Content-Type", "application/json");
res.send(
JSON.stringify({ state: "Failed", message: "Invalid API key" })
);
} else {
if (isFutureDate(result[0].expire)) {
// Has the key expired?
res.status(200);
res.cookie("betaToken", uuid.v4(), { signed: true });
res.redirect("/");
} else {
// Yes? Then no passing!
res.status(401);
res.setHeader("Content-Type", "application/json");
res.send(
JSON.stringify({ state: "Failed", message: "Expires API key" })
);
}
}
});
} else {
const data = fs.readFileSync("templates/betaLogin.eta.html", "utf8");
res.send(
Eta.render(data, {
siteTitel: "Pointsight - BetaLogin",
})
);
}
});
};

View File

@ -0,0 +1,19 @@
module.exports = function (app, jsonConfigGlobal, metaGlobals) {
const fs = require("fs");
const Eta = require("eta");
app.use(function (req, res, next) {
if(jsonConfigGlobal.maint){
const data = fs.readFileSync("templates/error/maint.eta", "utf8");
res.send(
Eta.render(data, {
desc: metaGlobals.desc,
siteTitel: metaGlobals.titlePrefx + "Maintanance work",
debug: {},
fontawesomeKey: jsonConfigGlobal.fontAwesome
})
);
}else{
next()
}
});
}

2
mysqlTables.sql Normal file
View File

@ -0,0 +1,2 @@
CREATE TABLE `apikeys` ( `id` BIGINT NOT NULL AUTO_INCREMENT , `apikey` VARCHAR(64) NOT NULL , `owner` VARCHAR(64) NOT NULL , `hosts` JSON NOT NULL , `created` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP , `expire` DATETIME NOT NULL , PRIMARY KEY (`id`));
CREATE TABLE `betatokens` ( `id` BIGINT NOT NULL AUTO_INCREMENT , `token` VARCHAR(64) NOT NULL , `owner` VARCHAR(64) NOT NULL, `created` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP , `expire` DATETIME NOT NULL , PRIMARY KEY (`id`));

64
package.json Normal file
View File

@ -0,0 +1,64 @@
{
"name": "pointsight",
"version": "1.1.1",
"description": "The ultimate knowledge map",
"main": "main.js",
"scripts": {
"makeJS": "tsc main.ts",
"lint": "eslint ./*.js && eslint ./*.ts",
"makeJSwatch": "tsc -w main.ts",
"start": "node tools/sqlMinifyer.js && tsc main.ts && node main.js",
"startDev": "node tools/sqlMinifyer.js && tsc main.ts && node --inspect main.js",
"preChecks": "npm outdated && npm audit",
"startBeforeMerge": "eslint . --ext .ts && tsc main.ts && node main.js",
"nodemon": "nodemon main.js",
"tailwind:css": "npx tailwindcss -i ./static/css/tailwindTemp.css -o ./static/css/tailwindInclude.css",
"checkLicense": "license-report --output=table --only=prod --config license-report-config.json"
},
"author": "[Project-Name-Here]",
"license": "LGPL-3.0",
"dependencies": {
"@sentry/node": "^6.16.0",
"@sentry/tracing": "^6.16.0",
"@themesberg/flowbite": "^1.2.0",
"autoprefixer": "^10.4.1",
"bent": "^7.3.12",
"body-parser": "^1.19.0",
"chalk": "^4.1.2",
"cli-progress": "^3.9.1",
"compression": "^1.7.4",
"cookie-parser": "^1.4.6",
"deep-email-validator": "^0.1.18",
"eta": "^1.12.3",
"express": "^4.17.2",
"express-hcaptcha": "^0.0.3",
"express-minify": "^1.0.0",
"express-winston": "^4.2.0",
"influx": "^5.9.2",
"license-report": "^4.5.0",
"log-symbols": "^5.1.0",
"mysql": "^2.18.1",
"postcss": "^8.4.5",
"postcss-cli": "^9.1.0",
"pp-which-country": "^1.0.2",
"safe-stable-stringify": "^2.3.1",
"tailwindcss": "^3.0.11",
"underscore": "^1.13.2",
"winston": "^3.3.3"
},
"devDependencies": {
"@types/node": "^16.11.19",
"@typescript-eslint/eslint-plugin": "^5.9.0",
"@typescript-eslint/parser": "^5.9.0",
"chai": "^4.3.4",
"chai-json": "^1.0.0",
"eslint": "^8.6.0",
"eslint-config-strongloop": "^2.1.0",
"inquirer": "^8.2.0",
"ora": "^6.0.1",
"prompts": "^2.4.2",
"supertest": "^6.1.6",
"typescript": "^4.5.4",
"uuid": "^8.3.2"
}
}

6
postcss.config.js Normal file
View File

@ -0,0 +1,6 @@
module.exports = {
plugins: [
require('tailwindcss'),
require('autoprefixer'),
]
}

76
proto/filterTest.js Normal file
View File

@ -0,0 +1,76 @@
const sensemap = require("../apiHandler/opensensemap/main");
const ttnantennas = require("../apiHandler/ttnantennas/main");
const autobahn = require("../apiHandler/autobahn/main");
const libs = require("../libs/metaHandler.lib.js");
const mods = [sensemap, ttnantennas, autobahn]; // Provided by main.ts in a real env.
const jsonData = {
filters: [
["tag", "webcam"],
["country", "DEU"],
],
conjunction: "AND",
}; // Provided by request in a real env
const uMods = mods.filter(function (elm) {
// Loops through all elements in a list and lets you decide if weather to keep 'em or not
let amount = 0;
for (fId in jsonData.filters) {
// Do this is each provided filter
const fi = jsonData.filters[fId]; // Gets the current filters name
switch (fi[0]) {
case "tag":
if (jsonData.conjunction == "OR") {
if (libs.returnTags(elm).includes(fi[1])) {
return elm;
} else {
break;
}
} else {
if (libs.returnTags(elm).includes(fi[1])) {
amount++;
}
break;
}
case "country":
if (jsonData.conjunction == "OR") {
if (libs.returnCountries(elm).includes(fi[1])) {
return elm;
} else {
break;
}
} else {
if (libs.returnCountries(elm).includes(fi[1])) {
amount++;
}
break;
}
}
}
if (jsonData.conjunction == "AND") {
if (amount == jsonData.filters.length) {
return elm;
}
}
});
console.log(uMods);
/**
* This things is sorting all given modules by a JSON array.
* You can supply the ˋfilterˋ array with a list of filters
* They are made up of ["metaData", "metaValue"]
* Another argument is ˋconjunctionˋ it can be either AND or OR
* OR means that any of the given filters should apply
* AND means that all filter HAVE to apply
*
* Current state
* This things can already handle OR requests, AND requests
* are not supported yet. I also use the /libs/metaHandler lib,
* at least I finnaly found a usecase for that waste of space.
*
* This is a small prototype which will be added into /routes/api.route.js
* at some point. But what I was doing before was almost production testing.
* Not good. I will need to test this further and throw some edge cases at it.
*/

58
proto/scrapeDemo.js Normal file
View File

@ -0,0 +1,58 @@
// Trying to scrape https://www.opentopia.com/hiddencam.php?p=n for data
// THIS IS A PROTOTYPE - DO NOT USE
// [Project-Name-Here] 2021
// Current state: Able to get cams from page, get location, unable to get live feed
process.exit(1)
const fs = require("fs");
const request = require("request");
function requestCamList(n){
request("http://www.opentopia.com/hiddencam.php?showmode=standard&country=%2A&seewhat=highlyrated&p="+n, { json: false }, (err, res2, body) => {
if (err) {
throw err;
}
let part = body.split("<body>")[1]
// part = part.split('<div style="clear:both">')[0]
//part = part.split('<ul class="camgrid camgrid3">')[1]
part = part.split("/webcam/")
part.shift()
for(h in part){
part[h] = part[h].replace("\t", "")
part[h] = part[h].split('target="_self">')[0].replace('"', "").split(">")[0]
console.log(part[h])
}
return(part)
});
}
function getMoreInfo(id){
const baseurl = "http://www.opentopia.com/webcam/"
const url = baseurl + id + "?viewmode=livevideo"
request(url, { json: false }, (err, res2, body) => {
if (err) {
throw err;
}
let result = {name: "", url:"", location:{lat:0,lng:0}}
let part = body.split("<body>")[1]
camPart = part.split("<div class=\"big\">")[1]
camPart = camPart.split('<img src="')[1]
camPart = camPart.split('"')[0]
result.url = camPart
part = part.split("Facility")[1]
part = part.split("Rating")[0]
part = part.split('<label class="right">')[1]
result.name = part.split("</label></div>")[0]
result.name = result.name.replaceAll("\t", "").replaceAll(" ", "")
part = part.split("<label class=\"right geo\">")[1]
part = part.split('<span class="latitude">')[1]
result.location.lat = parseFloat(part.split("</span>")[0])
part = part.split("<span class=\"longitude\">")[1]
result.location.lng = parseFloat(part.split("</span>")[0])
console.log(result)
});
}
console.log(getMoreInfo(6576))

320
routes/api.route.ts Normal file
View File

@ -0,0 +1,320 @@
module.exports = function (app, enabMods, con, otherContext) {
const wc = require('pp-which-country');
const _ = require("underscore");
const libs = require("../libs/metaHandler.lib");
const jLib = require("../libs/jsonHand.lib");
const stringify = require('safe-stable-stringify')
const mailValidator = require("deep-email-validator")
const tools = require("../functions");
// Returns all installed and non-limited modules
app.get("/api/allModules", function (req, res) {
res.setHeader("Content-Type", "application/json");
const allModules = [];
// Loop though all modules and sort all interesting ones into `allModules`
for (let i = 0; i < enabMods.length; i++) {
if (enabMods[i].getModuleMeta().limited == false) {
allModules.push(enabMods[i].getModuleMeta());
}
}
// Return all modules
res.send(allModules);
});
// Returns all tags with usages amounts
app.get("/api/allTags", function (req, res) {
res.setHeader("Content-Type", "application/json");
// Return all tags
const allTags = otherContext.tags
//allTags = allTags.sort(function(a,b) { return parseInt(a.z) - parseInt(b.z) } );
const response = []
for (const key in allTags) {
if (allTags.hasOwnProperty(key)) {
console.log(`${key}: ${allTags[key]}`);
response.push({key: key, amount: allTags[key]})
}
}
res.send(response);
});
// The report API endpoint
app.get("/api/internal/report", function (req, res) {
res.setHeader("Content-Type", "application/json");
// First check if all needed parameters are set
const point_id = req.query.point_id;
const mail = req.query.mail;
const reason = req.query.reason;
if (!point_id || !mail || !reason) {
res.send(JSON.stringify({
"status": "error",
"message": "Missing parameters"
}));
} else {
// Check if the point_id is valid
const splited = point_id.split("_");
if (splited.length != 2) {
res.send(JSON.stringify({
"status": "error",
"message": "Invalid point_id"
}));
} else {
let isValid = false;
for (let i = 0; i < enabMods.length; i++) {
if (enabMods[i].getModuleMeta().exactName == splited[0] && enabMods[i].getModuleMeta().limited == false) {
isValid = true;
}
}
if (isValid) {
// Check if the mail is valid
const mailValidationRes = mailValidator.validate({ email: mail, validateSMTP: false, validateTypo: false });
mailValidationRes.then(function (resp) {
if (resp.valid) {
const sqlQuery = "INSERT INTO `reports` (`point_id`, `mail`, `comment`) VALUES (?, ? , ?);";
con.query(sqlQuery, [point_id, mail, reason], function (err, rows, fields) {
tools.handleMysqlErrors(err, "API");
res.send(JSON.stringify({
"status": "success",
"message": "Report sent"
}));
});
} else {
res.send(JSON.stringify({
"status": "error",
"message": "Invalid mail",
"cause": resp
}));
}
})
} else {
res.send(JSON.stringify({
"status": "error",
"message": "Invalid point_id"
}));
}
}
}
});
app.get("/api/retrieve", function (req, res) {
// Make sure all parameters are there
if (
req.query.uid != undefined
) {
// All parameters are there
res.setHeader("Content-Type", "application/json");
const uid = req.query.uid;
let result = undefined;
console.warn("Retrieving data for UID: " + uid);
// Go thorugh all modules and find the one which should handle the request
for (let i = 0; i < enabMods.length; i++) {
if (enabMods[i].getModuleMeta().exactName == uid.split("_")[0] && enabMods[i].getModuleMeta().limited == false) {
result = enabMods[i].queryItem(uid);
console.warn("Found module: " + enabMods[i].getModuleMeta().exactName + " with result: " + result);
}
}
// Because DBs are slow the module only returns a Promise and we have to wait for it
Promise.all([result]).then(function (results) { res.send(results) });
} else {
// A parameter is missing
res.status(400);
res.setHeader("Content-Type", "application/json");
res.send(JSON.stringify({ state: "Failed", message: "Missing arguments" }));
}
});
app.get("/api/getPOI", function (req, res) {
// All parameters are there
res.setHeader("Content-Type", "application/json");
// Check if the bounds are valid so `lat;lng` is present
// First check if the `;` is present and .split() returns an array with 2 elements
const boundNorthEast = req.query.boundingEast.split(";");
const boundSouthWest = req.query.boundingWest.split(";");
let filterdModules = [];
if (boundNorthEast.length == 2 && boundSouthWest.length == 2) {
// Length is valid. Now check if the values are numbers and they are not infinte
if (_.isFinite(boundNorthEast[0]) && _.isFinite(boundNorthEast[1]) && _.isFinite(boundSouthWest[0]) && _.isFinite(boundSouthWest[1])) {
// Values are valid
const boundNorthEastLat = parseFloat(boundNorthEast[0]);
const boundNorthEastLng = parseFloat(boundNorthEast[1]);
const boundNorthEastLatLng = { lat: boundNorthEastLat, lng: boundNorthEastLng };
const boundSouthWestLat = parseFloat(boundSouthWest[0]);
const boundSouthWestLng = parseFloat(boundSouthWest[1]);
const boundSouthWestLatLng = { lat: boundSouthWestLat, lng: boundSouthWestLng };
// Get the country of both edges
// If the country is diffrent, set the country to *
const countryNorthEast = wc([boundNorthEastLng, boundNorthEastLat]);
let countrySouthWest = wc([boundSouthWestLng, boundSouthWestLat]);
if (countryNorthEast != countrySouthWest) {
// Country is the same
countrySouthWest = "*";
}
// Get all POIs in the given bounds
const usefullMods = []
for (let i = 0; i < enabMods.length; i++) {
const meta = enabMods[i].getModuleMeta()
// Is the country in the module's country list? Or is it "*"? Or did the system decide is doesnt matter
if ((meta.country.indexOf(countrySouthWest) > -1 || meta.country[0] == "*" || countrySouthWest == "*") && meta.limited == false) {
usefullMods.push(enabMods[i])
}
}
// JSON filter
if (jLib.IsJsonString(req.query.filter)) {
const jsonData = JSON.parse(req.query.filter);
filterdModules = usefullMods.filter(function (elm) {
// Loops through all elements in a list and lets you decide if weather to keep 'em or not
let amount = 0;
for (const fId in jsonData.filters) {
// Do this is each provided filter
const fi = jsonData.filters[fId]; // Gets the current filters name
switch (fi[0]) {
case "tag":
if (jsonData.conjunction == "OR") {
if (libs.returnTags(elm).includes(fi[1])) {
return elm;
} else {
break;
}
} else {
if (libs.returnTags(elm).includes(fi[1])) {
amount++;
}
break;
}
case "country":
if (jsonData.conjunction == "OR") {
if (libs.returnCountries(elm).includes(fi[1])) {
return elm;
} else {
break;
}
} else {
if (libs.returnCountries(elm).includes(fi[1])) {
amount++;
}
break;
}
case "source":
if (jsonData.conjunction == "OR") {
if (elm.getModuleMeta().exactName == fi[1]) {
return elm;
} else {
break;
}
} else {
if (elm.getModuleMeta().exactName == fi[1]) {
amount++;
}
break;
}
}
}
if (jsonData.conjunction == "AND") {
if (amount == jsonData.filters.length) {
return elm;
}
}
});
} else { // No filtering
filterdModules = enabMods;
}
// Create all promises
const allProms = []; // A list for all promises
for (let i = 0; i < filterdModules.length; i++) {
const module = filterdModules[i];
const temp = module.queryData(boundNorthEastLatLng, boundSouthWestLatLng, true); // Returns a promise of the results, because DBs are slowwwww
allProms.push(temp); // Put the promises into a list
}
// Wait for all promises and return them
Promise.all(allProms).then(function (result) {
const allResultsInOne = []
for (let i = 0; i < result.length; i++) {
for (let d = 0; d < result[i].length; d++) {
allResultsInOne.push(result[i][d])
}
}
res.send(stringify(allResultsInOne));
}).catch(function (error) {
console.error(error)
res.status(500).send(error); // We should probably not do this in a PROD env but FIXME: Only in DEV env
});
} else {
// Values are not valid
res.status(400);
res.send(JSON.stringify({ state: "Failed", message: "Bounding box is not valid" }));
return (0);
}
} else {
res.send(
JSON.stringify({ state: "Failed", message: "Invalid arguments" })
);
res.status(400);
return;
}
});
app.get("/api/getPOILocations", function (req, res) {
// Make sure all parameters are there
// All parameters are there
res.setHeader("Content-Type", "application/json");
if (_.isFinite(req.query.lat) && _.isFinite(req.query.lng) && _.isFinite(req.query.radius)) {
const lng = parseFloat(req.query.lng);
const lat = parseFloat(req.query.lat);
const radius = parseFloat(req.query.radius);
// Filter by usefull modules
const country = wc([lng, lat]); // Find out which country is at this location
const usefullMods = [];
// Go through all modules and find all fiting ones
for (let i = 0; i < enabMods.length; i++) {
const meta = enabMods[i].getModuleMeta()
if ((meta.country.indexOf(country) > -1 || meta.country[0] == "*") && meta.limited == false) {
usefullMods.push(enabMods[i])
}
}
const allProms = [];
for (let i = 0; i < usefullMods.length; i++) {
const module = usefullMods[i]
const temp = module.queryData(lat, lng, radius) // Returns a promise of the results
allProms.push(temp)
}
// Wait for all promises to be resolved
Promise.all(allProms).then(function (result) {
const allResultsInOne = []
for (let i = 0; i < result.length; i++) {
for (let d = 0; d < result[i].length; d++) {
allResultsInOne.push(result[i][d])
}
}
res.send(allResultsInOne);
}).catch(function (error) {
console.error(error)
res.status(500).send(error); // TODO: again only do it in DEV envs
});
} else {
res.send(
JSON.stringify({ state: "Failed", message: "Invalid arguments" })
);
res.status(400);
return;
}
});
}

58
routes/error.route.js Normal file
View File

@ -0,0 +1,58 @@
module.exports = function (app, jsonConfigGlobal, metaGlobals) {
const fs = require("fs");
const Eta = require("eta");
const util = require('util');
// Machine readable 404
app.get("/api/*", function (req, res) {
res.status(404);
res.setHeader("Content-Type", "application/json");
res.send(JSON.stringify({ state: "failed", message: "404-Unknown path" }));
});
// Users 404
app.get("*", function (req, res) {
res.status(404);
const data = fs.readFileSync("templates/error/404.eta", "utf8");
let debugData;
debugData = {
code: 404,
message: "Unknown path",
"Original url": req.originalUrl,
};
if (jsonConfigGlobal.env != "DEV") {
debugData = {};
}
res.send(
Eta.render(data, {
desc: metaGlobals.desc,
siteTitel: metaGlobals.titlePrefx + "404",
debug: debugData,
})
);
});
app.use(function (err, req, res, next) {
console.error(err.stack)
const data = fs.readFileSync("templates/error/serverError.eta", "utf8");
debugData = {trace: err.stack, avail: true}
if (jsonConfigGlobal.env != "DEV") {
debugData = {avail: false};
}
res.status(500)
res.send(
Eta.render(data, {
desc: metaGlobals.desc,
siteTitel: metaGlobals.titlePrefx + "500",
debug: debugData,
debugText: String(debugData.trace).replaceAll("\n", "<br>")
})
);
})
app.use(function (err, req, res, next) {
console.error(err.stack)
res.status(500)
res.send("Something went very wrong.")
})
};

36
routes/index.route.js Normal file
View File

@ -0,0 +1,36 @@
module.exports = function (app, metaGlobals, fontawesomeKey, mapboxAccessToken, config) {
const fs = require("fs");
const Eta = require("eta");
app.get("/", (req, res) => {
const data = fs.readFileSync("templates/index.eta.html", "utf8");
let indev = false;
if(config.ENV == "dev"){
indev = true
}
const errorPageContents = fs.readFileSync("templates/error/jserror.eta.html", "utf8").replace(new RegExp('\r?\n','g'), '<br />');
res.send(
Eta.render(data, {
desc: metaGlobals.desc,
siteTitel: metaGlobals.titlePrefx + "Map",
fontawesomeKey: fontawesomeKey,
mapboxAccessToken: mapboxAccessToken,
hereKey: config.here,
isInDev: indev,
errorPageContent: errorPageContents,
dsn: config.sentryDsn,
})
);
});
app.get("/about", (req, res) => {
const data = fs.readFileSync("templates/credits.eta.html", "utf8");
res.send(
Eta.render(data, {
desc: metaGlobals.desc,
siteTitel: metaGlobals.titlePrefx + "About",
fontawesomeKey: fontawesomeKey,
})
);
});
};

View File

@ -0,0 +1,102 @@
.leaflet-sidebar {
position: absolute;
height: 100%;
-webkit-box-sizing: border-box;
-moz-box-sizing: border-box;
box-sizing: border-box;
padding: 10px;
z-index: 2000; }
.leaflet-sidebar.left {
left: -500px;
transition: left 0.5s, width 0.5s;
padding-right: 0; }
.leaflet-sidebar.left.visible {
left: 0; }
.leaflet-sidebar.right {
right: -500px;
transition: right 0.5s, width 0.5s;
padding-left: 0; }
.leaflet-sidebar.right.visible {
right: 0; }
.leaflet-sidebar > .leaflet-control {
height: 100%;
width: 100%;
overflow: auto;
-webkit-overflow-scrolling: touch;
-webkit-box-sizing: border-box;
-moz-box-sizing: border-box;
box-sizing: border-box;
padding: 8px 24px;
font-size: 1.1em;
/*background: white;*/
box-shadow: 0 1px 7px rgba(0, 0, 0, 0.65);
-webkit-border-radius: 4px;
border-radius: 4px; }
.leaflet-touch .leaflet-sidebar > .leaflet-control {
box-shadow: none;
border: 2px solid rgba(0, 0, 0, 0.2);
background-clip: padding-box; }
@media (max-width: 767px) {
.leaflet-sidebar {
width: 100%;
padding: 0; }
.leaflet-sidebar.left.visible ~ .leaflet-left {
left: 100%; }
.leaflet-sidebar.right.visible ~ .leaflet-right {
right: 100%; }
.leaflet-sidebar.left {
left: -100%; }
.leaflet-sidebar.left.visible {
left: 0; }
.leaflet-sidebar.right {
right: -100%; }
.leaflet-sidebar.right.visible {
right: 0; }
.leaflet-sidebar > .leaflet-control {
box-shadow: none;
-webkit-border-radius: 0;
border-radius: 0; }
.leaflet-touch .leaflet-sidebar > .leaflet-control {
border: 0; } }
@media (min-width: 768px) and (max-width: 991px) {
.leaflet-sidebar {
width: 305px; }
.leaflet-sidebar.left.visible ~ .leaflet-left {
left: 305px; }
.leaflet-sidebar.right.visible ~ .leaflet-right {
right: 305px; } }
@media (min-width: 992px) and (max-width: 1199px) {
.leaflet-sidebar {
width: 390px; }
.leaflet-sidebar.left.visible ~ .leaflet-left {
left: 390px; }
.leaflet-sidebar.right.visible ~ .leaflet-right {
right: 390px; } }
@media (min-width: 1200px) {
.leaflet-sidebar {
width: 460px; }
.leaflet-sidebar.left.visible ~ .leaflet-left {
left: 460px; }
.leaflet-sidebar.right.visible ~ .leaflet-right {
right: 460px; } }
.leaflet-sidebar .close {
position: absolute;
right: 20px;
top: 20px;
width: 31px;
height: 31px;
color: #333;
font-size: 25px;
line-height: 1em;
text-align: center;
background: white;
-webkit-border-radius: 16px;
border-radius: 16px;
cursor: pointer;
z-index: 1000; }
.leaflet-left {
transition: left 0.5s; }
.leaflet-right {
transition: right 0.5s; }

View File

@ -0,0 +1,202 @@
L.Control.Sidebar = L.Control.extend({
includes: L.Evented.prototype || L.Mixin.Events,
options: {
closeButton: true,
position: 'left',
autoPan: true,
},
initialize: function (placeholder, options) {
L.setOptions(this, options);
// Find content container
var content = this._contentContainer = L.DomUtil.get(placeholder);
// Remove the content container from its original parent
if(content.parentNode != undefined){
content.parentNode.removeChild(content);
}
var l = 'leaflet-';
// Create sidebar container
var container = this._container =
L.DomUtil.create('div', l + 'sidebar ' + this.options.position);
// Style and attach content container
L.DomUtil.addClass(content, l + 'control');
container.appendChild(content);
// Create close button and attach it if configured
if (this.options.closeButton) {
var close = this._closeButton =
L.DomUtil.create('a', 'close', container);
close.innerHTML = '&times;';
}
},
addTo: function (map) {
var container = this._container;
var content = this._contentContainer;
// Attach event to close button
if (this.options.closeButton) {
var close = this._closeButton;
L.DomEvent.on(close, 'click', this.hide, this);
}
L.DomEvent
.on(container, 'transitionend',
this._handleTransitionEvent, this)
.on(container, 'webkitTransitionEnd',
this._handleTransitionEvent, this);
// Attach sidebar container to controls container
var controlContainer = map._controlContainer;
controlContainer.insertBefore(container, controlContainer.firstChild);
this._map = map;
// Make sure we don't drag the map when we interact with the content
var stop = L.DomEvent.stopPropagation;
var fakeStop = L.DomEvent._fakeStop || stop;
L.DomEvent
.on(content, 'contextmenu', stop)
.on(content, 'click', fakeStop)
.on(content, 'mousedown', stop)
.on(content, 'touchstart', stop)
.on(content, 'dblclick', fakeStop)
.on(content, 'mousewheel', stop)
.on(content, 'wheel', stop)
.on(content, 'scroll', stop)
.on(content, 'MozMousePixelScroll', stop);
return this;
},
removeFrom: function (map) {
//if the control is visible, hide it before removing it.
this.hide();
var container = this._container;
var content = this._contentContainer;
// Remove sidebar container from controls container
var controlContainer = map._controlContainer;
controlContainer.removeChild(container);
//disassociate the map object
this._map = null;
// Unregister events to prevent memory leak
var stop = L.DomEvent.stopPropagation;
var fakeStop = L.DomEvent._fakeStop || stop;
L.DomEvent
.off(content, 'contextmenu', stop)
.off(content, 'click', fakeStop)
.off(content, 'mousedown', stop)
.off(content, 'touchstart', stop)
.off(content, 'dblclick', fakeStop)
.off(content, 'mousewheel', stop)
.off(content, 'wheel', stop)
.off(content, 'scroll', stop)
.off(content, 'MozMousePixelScroll', stop);
L.DomEvent
.off(container, 'transitionend',
this._handleTransitionEvent, this)
.off(container, 'webkitTransitionEnd',
this._handleTransitionEvent, this);
if (this._closeButton && this._close) {
var close = this._closeButton;
L.DomEvent.off(close, 'click', this.hide, this);
}
return this;
},
isVisible: function () {
return L.DomUtil.hasClass(this._container, 'visible');
},
show: function () {
if (!this.isVisible()) {
L.DomUtil.addClass(this._container, 'visible');
if (this.options.autoPan) {
this._map.panBy([-this.getOffset() / 2, 0], {
duration: 0.5
});
}
this.fire('show');
}
},
hide: function (e) {
if (this.isVisible()) {
L.DomUtil.removeClass(this._container, 'visible');
if (this.options.autoPan) {
this._map.panBy([this.getOffset() / 2, 0], {
duration: 0.5
});
}
this.fire('hide');
}
if(e) {
L.DomEvent.stopPropagation(e);
}
},
toggle: function () {
if (this.isVisible()) {
this.hide();
} else {
this.show();
}
},
getContainer: function () {
return this._contentContainer;
},
getCloseButton: function () {
return this._closeButton;
},
setContent: function (content) {
var container = this.getContainer();
if (typeof content === 'string') {
container.innerHTML = content;
} else {
// clean current content
while (container.firstChild) {
container.removeChild(container.firstChild);
}
container.appendChild(content);
}
return this;
},
getOffset: function () {
if (this.options.position === 'right') {
return -this._container.offsetWidth;
} else {
return this._container.offsetWidth;
}
},
_handleTransitionEvent: function (e) {
if (e.propertyName == 'left' || e.propertyName == 'right')
this.fire(this.isVisible() ? 'shown' : 'hidden');
}
});
L.control.sidebar = function (placeholder, options) {
return new L.Control.Sidebar(placeholder, options);
};

View File

@ -0,0 +1,158 @@
$threshold-lg: 1200px;
$threshold-md: 992px;
$threshold-sm: 768px;
$width-lg: 460px;
$width-md: 390px;
$width-sm: 305px;
$width-xs: 100%;
$transition-duration: 0.5s;
@mixin border-radius($border-radius) {
-webkit-border-radius: $border-radius;
border-radius: $border-radius;
}
@mixin box-sizing($box-sizing) {
-webkit-box-sizing: $box-sizing;
-moz-box-sizing: $box-sizing;
box-sizing: $box-sizing;
}
@mixin widths($width) {
width: $width;
&.left.visible ~ .leaflet-left {
left: $width;
}
&.right.visible ~ .leaflet-right {
right: $width;
}
}
.leaflet-sidebar {
position: absolute;
height: 100%;
@include box-sizing(border-box);
padding: 10px;
z-index: 2000;
&.left {
left: -500px;
transition: left $transition-duration, width $transition-duration;
padding-right: 0;
&.visible {
left: 0;
}
}
&.right {
right: -500px;
transition: right $transition-duration, width $transition-duration;
padding-left: 0;
&.visible {
right: 0;
}
}
& > .leaflet-control {
height: 100%;
width: 100%;
overflow: auto;
-webkit-overflow-scrolling: touch;
@include box-sizing(border-box);
padding: 8px 24px;
font-size: 1.1em;
background: white;
box-shadow: 0 1px 7px rgba(0,0,0,0.65);
@include border-radius(4px);
.leaflet-touch & {
box-shadow: none;
border: 2px solid rgba(0,0,0,0.2);
background-clip: padding-box;
}
}
@media(max-width:$threshold-sm - 1px) {
@include widths($width-xs);
padding: 0;
&.left {
left: -$width-xs;
&.visible {
left: 0;
}
}
&.right {
right: -$width-xs;
&.visible {
right: 0;
}
}
& > .leaflet-control {
box-shadow: none;
@include border-radius(0);
.leaflet-touch & {
border: 0;
}
}
}
@media(min-width:$threshold-sm) and (max-width:$threshold-md - 1px) {
@include widths($width-sm);
}
@media(min-width:$threshold-md) and (max-width:$threshold-lg - 1px) {
@include widths($width-md);
}
@media(min-width:$threshold-lg) {
@include widths($width-lg);
}
.close {
position: absolute;
right: 20px;
top: 20px;
width: 31px;
height: 31px;
color: #333;
font-size: 25pt;
line-height: 1em;
text-align: center;
background: white;
@include border-radius(16px);
cursor: pointer;
// https://github.com/Turbo87/leaflet-sidebar/issues/36
z-index: 1000;
}
}
.leaflet-left {
transition: left $transition-duration;
}
.leaflet-right {
transition: right $transition-duration;
}

2
static/browserconfig.xml Normal file
View File

@ -0,0 +1,2 @@
<?xml version="1.0" encoding="utf-8"?>
<browserconfig><msapplication><tile><square70x70logo src="/favicon/ms-icon-70x70.png"/><square150x150logo src="/favicon/ms-icon-150x150.png"/><square310x310logo src="/favicon/ms-icon-310x310.png"/><TileColor>#ffffff</TileColor></tile></msapplication></browserconfig>

106
static/css/mainMap.css Normal file
View File

@ -0,0 +1,106 @@
body,
html {
background-color: white;
height: 100%;
margin: 0px;
}
.inspector {
margin: 0px;
height: 100vh;
width: 29%;
float: right;
position: fixed;
right: 0px;
top: 0px;
border: 1px solid rgba(97, 97, 97, 0.486);
border-radius: 2px;
margin: 5px;
padding: 4px;
overflow-y: scroll;
}
.fullheight {
height: 100%;
}
.map {
margin: 0px;
height: 100%;
width: 100%;
z-index: 1;
padding: 0px;
/*float: left;*/
}
.lds-ripple {
display: inline-block;
position: relative;
width: 80px;
height: 80px;
}
.lds-ripple div {
position: absolute;
border: 4px solid #fed;
opacity: 1;
border-radius: 50%;
animation: lds-ripple 1s cubic-bezier(0, 0.2, 0.8, 1) infinite;
}
.lds-ripple div:nth-child(2) {
animation-delay: -0.5s;
}
@keyframes lds-ripple {
0% {
top: 36px;
left: 36px;
width: 0;
height: 0;
opacity: 1;
}
100% {
top: 0px;
left: 0px;
width: 72px;
height: 72px;
opacity: 0;
}
}
.dropdown:hover .dropdown-menu {
display: block;
}
.reportBtn {
cursor: pointer;
font-size: 16px;
}
.animate-right {
position: relative;
animation: animateright 0.6s;
}
.animate-right-back {
position: relative;
animation: animaterightBack 0.6s;
}
@keyframes animateright {
from {
right: -300px;
opacity: 0;
}
to {
right: 0;
opacity: 1;
}
}
@keyframes animaterightBack {
from {
right: 0;
opacity: 1;
}
to {
right: -300px;
opacity: 0;
}
}

View File

@ -0,0 +1,3 @@
@tailwind base;
@tailwind components;
@tailwind utilities;

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.9 KiB

View File

@ -0,0 +1,2 @@
<?xml version="1.0" encoding="utf-8"?>
<browserconfig><msapplication><tile><square70x70logo src="/ms-icon-70x70.png"/><square150x150logo src="/ms-icon-150x150.png"/><square310x310logo src="/ms-icon-310x310.png"/><TileColor>#ffffff</TileColor></tile></msapplication></browserconfig>

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.3 KiB

BIN
static/favicon/favicon.ico Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 7.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.5 KiB

Some files were not shown because too many files have changed in this diff Show More