Skip to content

feat(docker): added workflows for building and pushing docker images #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 44 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
2e038dc
test: checking workflow trigger
Ansh-Sarkar Oct 6, 2022
9e6bdbb
Merge branch 'Samagra-Development:main' into main
Ansh-Sarkar Oct 14, 2022
7ed5281
feat(workflows): added docker build and push workflow
Ansh-Sarkar Oct 14, 2022
a1d5fd0
fix(workflows): build docs only on change
Ansh-Sarkar Oct 14, 2022
532e344
test(workflows): testing docker image build
Ansh-Sarkar Oct 14, 2022
66ba8ca
test(workflows): testing docker image build
Ansh-Sarkar Oct 14, 2022
fd23907
test(workflows): testing docker image build
Ansh-Sarkar Oct 14, 2022
270367e
Added
Ansh-Sarkar Oct 14, 2022
20d6b12
test
Ansh-Sarkar Oct 14, 2022
524523b
Added
Ansh-Sarkar Oct 14, 2022
796bd91
Added
Ansh-Sarkar Oct 14, 2022
4976f4a
Added
Ansh-Sarkar Oct 14, 2022
c918079
Added
Ansh-Sarkar Oct 14, 2022
e34104f
Added
Ansh-Sarkar Oct 14, 2022
ad6d8fe
Added
Ansh-Sarkar Oct 14, 2022
f15f4db
Added:
Ansh-Sarkar Oct 14, 2022
09ae9cc
added
Ansh-Sarkar Oct 14, 2022
2b8b56e
Added
Ansh-Sarkar Oct 14, 2022
d663af1
Added
Ansh-Sarkar Oct 14, 2022
af53436
Added
Ansh-Sarkar Oct 14, 2022
901c597
addded
Ansh-Sarkar Oct 14, 2022
33665fb
Added
Ansh-Sarkar Oct 14, 2022
5845af5
added
Ansh-Sarkar Oct 14, 2022
864efa7
feat(workflows): added workflow for form-manager package
Ansh-Sarkar Oct 15, 2022
eadb0fa
fix(workflows): removed run cmd
Ansh-Sarkar Oct 15, 2022
5b76196
test
Ansh-Sarkar Oct 15, 2022
28cf94a
fix(workflows): generalized github actors and workflow file
Ansh-Sarkar Oct 15, 2022
21f9215
fix(workflows): fixed lowecase issue for ghcr
Ansh-Sarkar Oct 15, 2022
c936fc4
added
Ansh-Sarkar Oct 15, 2022
846e920
Added
Ansh-Sarkar Oct 15, 2022
a674e64
Added
Ansh-Sarkar Oct 15, 2022
46af1f3
added
Ansh-Sarkar Oct 15, 2022
2b6b2c7
added
Ansh-Sarkar Oct 15, 2022
f68ebfb
checking workflow
Ansh-Sarkar Oct 15, 2022
1edc883
added
Ansh-Sarkar Oct 15, 2022
6c9bd81
Added
Ansh-Sarkar Oct 15, 2022
5b543e4
fix(workflows): fixed minor issues
Ansh-Sarkar Oct 15, 2022
9b5d9f3
fix(workflows): fixed minor naming issues
Ansh-Sarkar Oct 15, 2022
7a849a1
fix(workflows): reorder PR checks for triggering workflows
Ansh-Sarkar Oct 15, 2022
745af07
fix(config file): fix config file
Ansh-Sarkar Oct 19, 2022
e3c2988
test
Ansh-Sarkar Oct 20, 2022
88df3e2
Added
Ansh-Sarkar Oct 20, 2022
db76e69
fix(docs & docker): fixed readme and commented out sha files from docker
Ansh-Sarkar Oct 31, 2022
38dc639
test(workflows): removed overriding image build
Ansh-Sarkar Oct 31, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/deploy-to-github-pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ name: Deploy static content to Pages
on:
# Runs on pushes targeting the default branch
push:
paths:
- docs/**
branches: [main]

# Allows you to run this workflow manually from the Actions tab
Expand Down
23 changes: 23 additions & 0 deletions .github/workflows/docker-build-push.yml

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not Remove it if everything is commented out?

Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# name: publish
# on:
# push:
# paths:
# - packages/enketo-express/**
# branches: [main]
# jobs:
# publish-hello-docker-image:
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v2
# - name: Login to GitHub Container Registry
# uses: docker/login-action@v1
# with:
# registry: ghcr.io
# username: ${{ github.actor }}
# password: ${{ secrets.GITHUB_TOKEN }}
# - name: Build the hello-docker Docker image
# run: |
# cd packages/enketo-express
# docker build . --tag ghcr.io/ansh-sarkar/enketo-express:latest
# docker run ghcr.io/ansh-sarkar/enketo-express:latest
# docker push ghcr.io/ansh-sarkar/enketo-express:latest
29 changes: 29 additions & 0 deletions .github/workflows/enketo-express-docker-build-push-ghcr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Enketo Express Publish
on:
push:
paths:
- packages/enketo-express/**
tags:
- '*'
branches: [main]
jobs:
publish-enketo-express-docker-image:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: set lower case owner name
run: |
echo "OWNER_LC=${OWNER,,}" >>${GITHUB_ENV}
env:
OWNER: '${{ github.repository_owner }}'
- name: Login to GitHub Container Registry
uses: docker/login-action@v1
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build enketo-express Docker image
run: |
cd packages/enketo-express
docker build . --tag ghcr.io/$OWNER_LC/enketo-express:latest
docker push ghcr.io/$OWNER_LC/enketo-express:latest
29 changes: 29 additions & 0 deletions .github/workflows/form-manager-docker-build-push-ghcr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Form Manager Publish
on:
push:
paths:
- packages/form-manager/**
tags:
- '*'
branches: [main]
jobs:
publish-form-manager-docker-image:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: set lower case owner name
run: |
echo "OWNER_LC=${OWNER,,}" >>${GITHUB_ENV}
env:
OWNER: '${{ github.repository_owner }}'
- name: Login to GitHub Container Registry
uses: docker/login-action@v1
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build form-manager Docker image
run: |
cd packages/form-manager
docker build . --tag ghcr.io/$OWNER_LC/form-manager:latest
docker push ghcr.io/$OWNER_LC/form-manager:latest
120 changes: 34 additions & 86 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,114 +1,62 @@
<p align="center">
<h1>Enketo - ODK Backend</h1>
</p>
# Introduction to Workflow

## About :open_book:
Workflow is an Open Source project aimed towards the creation of data flow models using config files thereby allowing you to easily create and interact with stateful applications with minimum setup.

Enketo has been used all around the world for a wide range of uses from needs assessments in humanitarian aid, to raising historical awareness, to clinical research, to election monitoring.
We appreciate your interest in the project. Here are a few resources which you might find useful in order to get started or dive deeper into its architecture.

## Features :dart:
[Introduction to Workflow](https://samagra-development.github.io/workflow/) 👈 You are here

- [x] Can connect to your custom backend
- [ ] Have the edit api in case user wants to edit submitted response
- [ ] Api for the customize change in XML of form
- [x] Have beautiful themes and widgets
- [x] Are printer-friendly
- [x] Can use very powerful skip and validation logic
- [x] Run on any device, mobile or desktop, as long as it has a fairly modern browser
👉󠀠󠀠󠀠 󠀠󠀠󠀠󠀠󠀠[Getting Started](https://samagra-development.github.io/workflow/category/getting-started) <br/>
👉 [Guides](https://samagra-development.github.io/workflow/category/guides) <br/>
👉 [Advanced Guides](https://samagra-development.github.io/workflow/category/advanced-guides) <br/>
👉 [Future Roadmaps](https://samagra-development.github.io/workflow/category/future-roadmaps) <br/>
👉 [Community](https://samagra-development.github.io/workflow/category/community)

## Use Case :rocket:
> If you would like to contribute to this project, kindly refer to the **Contributions** Page under the **Community** category. Kindly make

![Screencast from 03-02-22 05_31_50 PM IST](https://user-images.githubusercontent.com/56133783/152341126-ac03baa3-258c-473a-956a-d0973682234a.gif)
<br>

## Requirements :scroll:
# Setup and Installation

1. Your machine should have [Yarn](https://classic.yarnpkg.com/en/docs/install/#windows-stable) or [Npm](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm), [Docker](https://docs.docker.com/get-docker/), [Python](https://www.python.org/downloads/) installed.
## System Requirements

*Note: Preferable Npm version (6.14.15) and Node version(v14.18.1) and Ubuntu OS version(18.04)**
The primary requirements of the project include [**NodeJS**](https://nodejs.org/en/) and [**Docker**](https://www.docker.com/). Multiple ```node``` versions have been used and hence it is recommended that you have the latest version of ```NVM - Node Version Manager``` installed on your computer. A ```.nvmrc``` file will be added to every directory in order to make sure that the current ```node``` version is being used to build it.

2. Check the node and npm version by running following commands.
> **Recommended :** NPM version (6.14.15) and Node versio (v14.18.1) and Ubuntu OS version (18.04)
> Your machine should have [Yarn](https://yarnpkg.com/) / [NPM](https://www.npmjs.com/) & [Python](https://www.python.org/) installed.

```sh
node -v
npm -v
```

## Installation Steps :walking:

### 1. Fork it :fork_and_knife:

You can get your own fork/copy of [enketo](https://github.com/Samagra-Development/enketo) by using the <kbd><b>Fork</b></kbd> button.

### 2. Clone it :busts_in_silhouette:
> You can check your ```node``` and ```npm``` versions by running the following commands

You need to clone (download) it to a local machine using

```sh
git clone https://github.com/Your_Username/enketo.git
```

> This makes a local copy of the repository in your machine.

Once you have cloned the `enketo` repository in GitHub, move to that folder first using the change directory command.

```sh
# This will change directory to a folder FOSSologyUI
cd enketo
node -v
npm -v
```

Move to this folder for all other commands.

### 3. Set it up :arrow_up:
## Installation

Run the following commands to see that _your local copy_ has a reference to _your forked remote repository_ in GitHub :octocat:
In order to make the process of getting started with the project as easy and seamless as possible, we have a single command setup for enabling fast and convenient setup.

```sh
git remote -v
origin https://github.com/Your_Username/enketo.git (fetch)
origin https://github.com/Your_Username/enketo.git (push)
```

### 4. Run it :checkered_flag:

```sh
cd enketo-express
docker run --name enketo-redis-main -p 6379:6379 -d redis
docker run --name enketo-redis-cache -p 6380:6379 -d redis
npm install
npm i -g grunt
grunt develop
```
👉 Go ahead and fork the repository to get your own copy by clicking on the ```Fork``` button.

```sh
cd ../enketo-core
npm install
npm start
```
👉 Clone the repository and select the **workflow** directory as the current directory.

```sh
cd ../enketo-transformer
npm install
npm start
```bash
git clone https://github.com/Samagra-Development/workflow
cd workflow
```

```sh
cd ../forms
python3 -m http.server
👉 Setup and all start packages and applications by running this single command
```bash
npm run start
```

You can preview your form on http://localhost:8005/preview?xform=http://localhost:8080/getForm/SOE

### 5. For integrating your backend

Customize the [submission url](https://github.com/Samagra-Development/enketo/blob/main/enketo-express/public/js/src/module/connection.js#L150) with your hosted backend.
This command should setup and initialize all the different packages on which the project depends on and also start the sample application for you to experiment with.

## Future Enhancements :rocket:
👉 The repository is structured as a **monorepo** managed using [**Turborepo**](https://turborepo.org/) in order to cache and speed up builds, create pipelines and enabling the concurrent execution of scripts, all of which result in an improved Developer Experience (DX). Active work continues on this and progress can be tracked [**here**](https://github.com/Samagra-Development/workflow)

- [Encrypting query parameters on rendering the enketo form. ](https://github.com/Samagra-Development/enketo/issues/1)
- Writing a API for changing an XML form directly from it.
- add npm scripts for install
> Make sure you have ```Docker``` installed on your system as some of the packages depend on underlying docker containers. The team is currently planning to shift all servers to containers. You can track the progress on this migration [here](https://github.com/Samagra-Development/workflow/issues/14)

We are trying to an inversion of control here.
<hr>

### For Get API

Expand Down Expand Up @@ -258,5 +206,5 @@ State for prefilling, sideEffect.
TODO: Add details on the specifications

## Possible Attack Vectors
1. XSS (High Priority) - Simple form
1. XSS (High Priority) - Simple form.
2. SQL Injection (High Priority) - needs to be fixed.
2 changes: 2 additions & 0 deletions packages/enketo-express/setup/docker/create_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ def get_or_create_encryption_key():
def create_config():

CONFIG_FILE_PATH = os.path.join(PROJECT_ROOT_PATH, 'config/config.json')
if not os.path.isfile(CONFIG_FILE_PATH):
CONFIG_FILE_PATH = os.path.join(PROJECT_ROOT_PATH, 'config/default-config.json')
if not os.path.isfile(CONFIG_FILE_PATH):
raise EnvironmentError('No Enketo Express configuration found at `{}`.'.format(CONFIG_FILE_PATH))
else:
Expand Down
6 changes: 3 additions & 3 deletions packages/enketo-express/setup/docker/start.sh

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the Commented lines are not useful in the future it's better to remove them isn't it?

Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@ CHECKSUM_DIR_PATH="${ENKETO_SRC_DIR}/checksum"

CONFIG_FILE="config.json"
CONFIG_FILE_PATH="${ENKETO_SRC_DIR}/config/${CONFIG_FILE}"
SHA1SUM_CONFIG_FILE_PATH="${CHECKSUM_DIR_PATH}/${CONFIG_FILE}.sha1"
# SHA1SUM_CONFIG_FILE_PATH="${CHECKSUM_DIR_PATH}/${CONFIG_FILE}.sha1"

LAST_BUILD_COMMIT_FILE="last_build_commit.txt"
LAST_BUILD_COMMIT_FILE_PATH="${CHECKSUM_DIR_PATH}/${LAST_BUILD_COMMIT_FILE}"

RUN_GRUNT=0
# Compare config version
sha1sum --status -c "${SHA1SUM_CONFIG_FILE_PATH}" || RUN_GRUNT=1
# sha1sum --status -c "${SHA1SUM_CONFIG_FILE_PATH}" || RUN_GRUNT=1

# Compare commit version
CURRENT_COMMIT=$(git rev-parse HEAD) # Get current commit
Expand All @@ -46,7 +46,7 @@ if [ "$RUN_GRUNT" == 1 ]; then
echo "Saving current commit..."
echo $CURRENT_COMMIT > ${LAST_BUILD_COMMIT_FILE_PATH}
echo "Saving config hash..."
sha1sum ${CONFIG_FILE_PATH} > "${SHA1SUM_CONFIG_FILE_PATH}"
# sha1sum ${CONFIG_FILE_PATH} > "${SHA1SUM_CONFIG_FILE_PATH}"
echo "Creating .not-empty files"
touch $NOT_EMPTY_JS_PATH
touch $NOT_EMPTY_CSS_PATH
Expand Down
22 changes: 22 additions & 0 deletions packages/form-manager/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Base image
FROM node:14

# Create app directory
WORKDIR /usr/src/app

# A wildcard is used to ensure both package.json AND package-lock.json are copied
COPY package*.json ./

# Install app dependencies
RUN npm install

# Bundle app source
COPY . .

# Creates a "dist" folder with the production build
RUN npm run build

# Start the server using the production build
CMD [ "node", "dist/main.js" ]

EXPOSE 3002/tcp
2 changes: 1 addition & 1 deletion packages/form-manager/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<p align="center">
<a href="http://nestjs.com/" target="blank"><img src="https://nestjs.com/img/logo-small.svg" width="200" alt="Nest Logo" /></a>
<a href="http://nestjs.com/" target="_blank"><img src="https://nestjs.com/img/logo-small.svg" width="200" alt="Nest Logo" /></a>
</p>

[circleci-image]: https://img.shields.io/circleci/build/github/nestjs/nest/master?token=abc123def456
Expand Down