feat: flyway runner construct (#157)

* add flyway construct

* fix file name

* add unit test for log retention option

* put back integ test file

* fix modified projen

* force node12

* chore: self mutation

* remove classes

* chore: self mutation

* remove classes

* chore: self mutation

* Move to pre bundle lambda and custom resource provider

* remove tests that don't work with pre bundle

* move to projen 0.33.1

* revert projen version related changes

* Add Java 11 to github build action

* fix jar copy

* chore: self mutation

* Add special LICENSE entry for FlywayLambda code

Respect MIT license requirement.

* chore: self mutation

Co-authored-by: Chazal <[email protected]>
Co-authored-by: Automation <[email protected]>

Author: 3 people

.github/workflows/build.yml

@@ -17,6 +17,11 @@ jobs:
         uses: actions/setup-node@v2
         with:
           node-version: '12'
+      - name: Setup Java 11
+        uses: actions/setup-java@v2
+        with:
+          distribution: 'zulu' # OpenJDK
+          java-version: '11'
       - name: Install dependencies
         run: cd core && yarn install --check-files --frozen-lockfile
       - name: Set git identity

CONTRIB_FAQ.md

@@ -204,13 +204,15 @@ To prevent this issue, we need to use `path.join(__dirname, 'resources/first.sql
 
 Place your Lambda function code under `<construct-folder>/resources/lambdas/<lambdan-function-name>` folder. Projen will detect the new folder and bundle that during `npx projen build`. 
 
+#### For python
+
 It will copy all files to the same path in `lib` folder and install Python dependencies on any folder with `requirements.txt`. Our package will have all dependencies bundle and consumers do not have to install dependencies by themselves.
 
 In the construct, create a Lambda function with `PreBundledFunction` construct to use the prebundled version. All of the parameter is the same as `lambda.Function`. The only difference is passing `codePath` prop instead of `code` with a relative path from `core/src`. The construct will use the right path with all dependencies when consumer is building. 
 
 Here's an example:
 
-```
+```typescript
 new PreBundledFunction(this, 'helloWordNumpy', {
     runtime: Runtime.PYTHON_3_8,
     codePath: '<construct-folder>/resources/lambdas/<lambdan-function-name>',
@@ -219,13 +221,28 @@ new PreBundledFunction(this, 'helloWordNumpy', {
     timeout: Duration.seconds(30),
 });
 ```
+
+### For Java (Gradle)
+
+It will copy all files to the same path in `lib` folder and call `./gradlew build shadowJar`  on any folder with `build.gradle`. Our package will have all dependencies bundle and consumers do not have to install java or dependencies by themselves.
+
+Here's an example:
+
+```typescript
+new PreBundledFunction(this, 'runner', {
+      codePath: path.join(__dirname.split('/').slice(-1)[0], './resources/flyway-lambda/build/libs/flyway-all.jar'),
+      handler: 'com.geekoosh.flyway.FlywayCustomResourceHandler::handleRequest',
+      runtime: lambda.Runtime.JAVA_11,
+    });
+```
+
 ### Why do we prebundle Python code for Lambda function?
 
 Most of the data engineering code are written in Python. Even our CDK constructs are in TypeScript, we still use Python a lot in our Lambda functions. 
 
 Normally, construct provider will **defer bundling step to construct consumers**. The provider only packages Python files and `requirements.txt` and distribute them. Consumers have to install and compile dependencies by themselves. The code looks like this:
 
-```
+```typescript
 new PythonFunction(this, 'functionName', {
     runtime: Runtime.PYTHON_3_8,
     entry: '<construct-folder>/resources/lambdas/<function-name>',

LICENSE

@@ -150,3 +150,8 @@ Section 8 – Interpretation.
      c.	No term or condition of this Public License will be waived and no failure to comply consented to unless expressly agreed to by the Licensor.
 
      d.	Nothing in this Public License constitutes or may be interpreted as a limitation upon, or waiver of, any privileges and immunities that apply to the Licensor or You, including from the legal processes of any jurisdiction or authority.
+
+
+** FlywayLambdaService (core/src/db-schema-manager/resources/flyway-lambda) - https://github.com/Geekoosh/flyway-lambda
+
+Copyright (c) 2021 Assaf Kamil

core/.gitattributes

@@ -55,6 +55,12 @@ const project = new AwsCdkConstructLibrary({
     '@aws-cdk/aws-stepfunctions-tasks',
     '@aws-cdk/aws-events',
     '@aws-cdk/aws-events-targets',
+    '@aws-cdk/aws-s3-deployment',
+    '@aws-cdk/aws-ec2',
+    '@aws-cdk/aws-redshift',
+    '@aws-cdk/aws-secretsmanager',
+    '@aws-cdk/aws-s3-assets',
+    '@aws-cdk/assertions',
   ],
   bundledDeps: [
     'xmldom@github:xmldom/xmldom#0.7.0',
@@ -119,11 +125,29 @@ for (const dirPath of findAllPythonLambdaDir('src')) {
   pipInstallTask.exec(pipCmd);
 }
 
+/**
+ * Task to build java lambda jar with gradle
+ */
+const gradleBuildTask = project.addTask('gradle-build', {
+  description: './gradlew shadowJar all folders in lib that has requirements.txt',
+});
+
+for (const dirPath of findAllGradleLambdaDir('src')) {
+  console.log('loop over gradle dir');
+  // Assume that all folders with 'requirements.txt' have been copied to lib
+  // by the task 'copy-resources'
+  const dirPathInLib = dirname(dirPath.replace('src', 'lib'));
+  const gradleCmd = `cd ${dirPathInLib} && ./gradlew shadowJar && cp build/libs/*.jar ./ 2> /dev/null`;
+
+  gradleBuildTask.exec(gradleCmd);
+}
+
 /**
  * Run `copy-resources` and `pip-install` as part of compile
  */
 project.compileTask.exec('npx projen copy-resources');
 project.compileTask.exec('npx projen pip-install');
+project.compileTask.exec('npx projen gradle-build');
 
 /**
  * Find all directory that has a Python package.
@@ -139,4 +163,19 @@ function findAllPythonLambdaDir(rootDir) {
   });
 }
 
+/**
+ * Find all directory that has a gradle package.
+ * Assume that they have build.gradle
+ *
+ * @param rootDir Root directory to begin finding
+ * @returns Array of directory paths
+ */
+function findAllGradleLambdaDir(rootDir) {
+  console.log('findAllGradleLambdaDir');
+
+  return glob.sync(`${rootDir}/**/build.gradle`).map((pathWithGradle) => {
+    return pathWithGradle;
+  });
+}
+
 project.synth();