Skip to content

Commit ba56ef5

Browse files
committed
Setup monorepo with lerna, #103
1 parent 8671d98 commit ba56ef5

File tree

189 files changed

+6315
-2375
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

189 files changed

+6315
-2375
lines changed

.gitignore

+3
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,6 @@ config.json
99
config/*.json
1010
!config/config-defaults.json
1111
!config/config-example*.json
12+
13+
# Ignore dev environment files
14+
.idea

.gitlab-ci.yml

-17
This file was deleted.

.travis.yml

+6-3
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,12 @@ language: node_js
22
node_js:
33
- "6"
44
- "8"
5-
- "node"
5+
- "10"
6+
- "12"
7+
install: yarn install --pure-lockfile
68
script:
7-
- npm run lint
8-
- npm test
9+
- yarn run lint
10+
- yarn test
911
env:
1012
- CXX=g++-4.8
1113
addons:
@@ -15,5 +17,6 @@ addons:
1517
packages:
1618
- g++-4.8
1719
cache:
20+
yarn: true
1821
directories:
1922
- node_modules

AUTHORS

+1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
Ruben Verborgh <[email protected]> (http://ruben.verborgh.org/)
22
Miel Vander Sande <[email protected]>
3+
Ruben Taelman <[email protected]>

LICENSE.txt

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
The MIT License (MIT)
22

3-
Copyright © 2013–2016 Ruben Verborgh, Miel Vander Sande
3+
Copyright © 2013–now Ruben Verborgh, Miel Vander Sande, Ruben Taelman
44
Ghent University – imec, Belgium
55

66
Permission is hereby granted, free of charge, to any person obtaining a copy

README.md

+27-156
Original file line numberDiff line numberDiff line change
@@ -29,178 +29,49 @@ Each Triple Pattern Fragment offers:
2929

3030
An example server is available at [data.linkeddatafragments.org](http://data.linkeddatafragments.org/).
3131

32+
TODO: briefly explain configurations
3233

33-
## Install the server
34-
35-
This server requires [Node.js](http://nodejs.org/) 4.0 or higher
36-
and is tested on OSX and Linux.
37-
To install, execute:
38-
```bash
39-
$ [sudo] npm install -g ldf-server
40-
```
41-
42-
43-
## Use the server
44-
45-
### Configure the data sources
46-
47-
First, create a configuration file `config.json` similar to `config/config-example.json`,
48-
in which you detail your data sources.
49-
For example, this configuration uses an [HDT file](http://www.rdfhdt.org/)
50-
and a [SPARQL endpoint](http://www.w3.org/TR/sparql11-protocol/) as sources:
51-
```json
52-
{
53-
"title": "My Linked Data Fragments server",
54-
"datasources": {
55-
"dbpedia": {
56-
"title": "DBpedia 2014",
57-
"type": "HdtDatasource",
58-
"description": "DBpedia 2014 with an HDT back-end",
59-
"settings": { "file": "data/dbpedia2014.hdt" }
60-
},
61-
"dbpedia-sparql": {
62-
"title": "DBpedia 3.9 (Virtuoso)",
63-
"type": "SparqlDatasource",
64-
"description": "DBpedia 3.9 with a Virtuoso back-end",
65-
"settings": { "endpoint": "https://dbpedia.org/sparql", "defaultGraph": "http://dbpedia.org" }
66-
}
67-
}
68-
}
69-
```
70-
71-
The following sources are supported out of the box:
72-
- HDT files ([`HdtDatasource`](https://github.com/LinkedDataFragments/Server.js/blob/master/lib/datasources/HdtDatasource.js) with `file` setting)
73-
- N-Triples documents ([`NTriplesDatasource`](https://github.com/LinkedDataFragments/Server.js/blob/master/lib/datasources/NTriplesDatasource.js) with `url` setting)
74-
- Turtle documents ([`TurtleDatasource`](https://github.com/LinkedDataFragments/Server.js/blob/master/lib/datasources/TurtleDatasource.js) with `url` setting)
75-
- N-Quads documents ([`NQuadsDatasource`](https://github.com/LinkedDataFragments/Server.js/blob/master/lib/datasources/NQuadsDatasource.js) with `url` setting)
76-
- TriG documents ([`TrigDatasource`](https://github.com/LinkedDataFragments/Server.js/blob/master/lib/datasources/TrigDatasource.js) with `url` setting)
77-
- JSON-LD documents ([`JsonLdDatasource`](https://github.com/LinkedDataFragments/Server.js/blob/master/lib/datasources/JsonLdDatasource.js) with `url` setting)
78-
- SPARQL endpoints ([`SparqlDatasource`](https://github.com/LinkedDataFragments/Server.js/blob/master/lib/datasources/SparqlDatasource.js) with `endpoint` and optionally `defaultGraph` settings)
79-
80-
Support for new sources is possible by implementing the [`Datasource`](https://github.com/LinkedDataFragments/Server.js/blob/master/lib/datasources/Datasource.js) interface.
81-
82-
### Start the server
83-
84-
After creating a configuration file, execute
85-
```bash
86-
$ ldf-server config.json 5000 4
87-
```
88-
Here, `5000` is the HTTP port on which the server will listen,
89-
and `4` the number of worker processes.
90-
91-
Now visit `http://localhost:5000/` in your browser.
34+
**If you just want to use this server, have a look at these default configurations**:
35+
* TODO
9236

93-
### Reload running server
37+
This repository should be used by LDF Server module **developers** as it contains multiple LDF Server modules that can be composed.
38+
This repository is managed as a [monorepo](https://github.com/babel/babel/blob/master/doc/design/monorepo.md)
39+
using [Lerna](https://lernajs.io/).
9440

95-
You can reload the server without any downtime
96-
in order to load a new configuration or version.
97-
<br>
98-
In order to do this, you need the process ID of the server master process.
99-
<br>
100-
One possibility to obtain this are the server logs:
101-
```bash
102-
$ bin/ldf-server config.json
103-
Master 28106 running.
104-
Worker 28107 running on http://localhost:3000/.
105-
```
106-
107-
If you send the server a `SIGHUP` signal:
108-
```bash
109-
$ kill -s SIGHUP 28106
110-
```
111-
it will reload by replacing its workers.
112-
113-
Note that crashed or killed workers are always replaced automatically.
114-
115-
### _(Optional)_ Set up a reverse proxy
116-
117-
A typical Linked Data Fragments server will be exposed
118-
on a public domain or subdomain along with other applications.
119-
Therefore, you need to configure the server to run behind an HTTP reverse proxy.
120-
<br>
121-
To set this up, configure the server's public URL in your server's `config.json`:
122-
```json
123-
{
124-
"title": "My Linked Data Fragments server",
125-
"baseURL": "http://data.example.org/",
126-
"datasources": { }
127-
}
128-
```
129-
Then configure your reverse proxy to pass requests to your server.
130-
Here's an example for [nginx](http://nginx.org/):
131-
```nginx
132-
server {
133-
server_name data.example.org;
134-
135-
location / {
136-
proxy_pass http://127.0.0.1:3000$request_uri;
137-
proxy_set_header Host $http_host;
138-
proxy_pass_header Server;
139-
}
140-
}
141-
```
142-
Change the value `3000` into the port on which your Linked Data Fragments server runs.
143-
144-
If you would like to proxy the data in a subfolder such as `http://example.org/my/data`,
145-
modify the `baseURL` in your `config.json` to `"http://example.org/my/data"`
146-
and change `location` from `/` to `/my/data` (excluding a trailing slash).
147-
148-
### _(Optional)_ Running under HTTPS
149-
150-
HTTPS can be enabled in two ways: natively by the server, or through a proxy (explained above).
151-
152-
With native HTTPS, the server will establish the SSL layer. Set the following values in your config file to enable this:
41+
## Install the server
15342

154-
{
155-
"protocol": "https",
156-
"ssl": {
157-
"keys" : {
158-
"key": "./private-key-server.key.pem",
159-
"ca": ["./root-ca.crt.pem"],
160-
"cert": "./server-certificate.crt.pem"
161-
}
162-
}
163-
}
164-
165-
If `protocol`is not specified, it will derive the protocol from the `baseURL`. Hence, HTTPS can also be enabled as such:
43+
TODO: basic installation, and refer to packages for more details
16644

167-
{
168-
"baseURL": "https://data.example.org/",
169-
"ssl": {
170-
"keys" : {
171-
"key": "./private-key-server.key.pem",
172-
"ca": ["./root-ca.crt.pem"],
173-
"cert": "./server-certificate.crt.pem"
174-
}
175-
}
176-
}
45+
## Use the server
17746

178-
If you decide to let a proxy handle HTTPS, use this configuration to run the server as `http`, but construct links as `https` (so clients don't break):
47+
TODO: basic usage, and refer to packages for more details
17948

180-
{
181-
"protocol": "http",
182-
"baseURL": "https://data.example.org/",
183-
}
49+
## Development Setup
18450

51+
If you want to develop new features
52+
or use the (potentially unstable) in-development version,
53+
you can set up a development environment for this server.
18554

186-
### _(Optional)_ Running in a Docker container
55+
LDF Server requires [Node.JS](http://nodejs.org/) 6.0 or higher and the [Yarn](https://yarnpkg.com/en/) package manager.
56+
LDF Server is tested on OSX, Linux and Windows.
18757

188-
If you want to rapidly deploy the server as a microservice, you can build a [Docker](https://www.docker.com/) container as follows:
58+
This project can be setup by cloning and installing it as follows:
18959

19060
```bash
191-
$ docker build -t ldf-server .
192-
```
193-
After that, you can run your newly created container:
194-
```bash
195-
$ docker run -p 3000:3000 -t -i --rm -v $(pwd)/config.json:/tmp/config.json ldf-server /tmp/config.json
61+
$ git clone https://github.com/LinkedDataFragments/Server.js.git
62+
$ cd comunica
63+
$ yarn install
19664
```
19765

198-
### _(Optional)_ Host historical version of datasets
66+
**Note: `npm install` is not supported at the moment, as this project makes use of Yarn's [workspaces](https://yarnpkg.com/lang/en/docs/workspaces/) functionality**
19967

200-
You can [enable the Memento protocol](https://github.com/LinkedDataFragments/Server.js/wiki/Configuring-Memento) to offer different versions of an evolving dataset.
68+
This will install the dependencies of all modules, and bootstrap the Lerna monorepo.
69+
After that, all [LDF Server packages](https://github.com/LinkedDataFragments/Server.js/tree/master/packages) are available in the `packages/` folder
70+
and can be used in a development environment.
20171

202-
## License
203-
The Linked Data Fragments server is written by [Ruben Verborgh](http://ruben.verborgh.org/).
72+
Furthermore, this will add [pre-commit hooks](https://www.npmjs.com/package/pre-commit)
73+
to build, lint and test.
74+
These hooks can temporarily be disabled at your own risk by adding the `-n` flag to the commit command.
20475

20576
## License
20677
The Linked Data Fragments client is written by [Ruben Verborgh](http://ruben.verborgh.org/) and colleagues.

lerna.json

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
{
2+
"lerna": "3.4.0",
3+
"command": {
4+
"publish": {
5+
"ignoreChanges": [
6+
"*.md"
7+
],
8+
"message": "Bump to release version %s",
9+
"allowBranch": "master",
10+
"npmClient": "npm"
11+
}
12+
},
13+
"packages": [
14+
"packages/*"
15+
],
16+
"useWorkspaces": true,
17+
"version": "2.2.5",
18+
"loglevel": "success",
19+
"registry": "https://registry.npmjs.org/",
20+
"npmClient": "yarn"
21+
}

0 commit comments

Comments
 (0)