Skip to content

Support for uploading to Azure File Storage #128

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 40 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
0153079
Keep progress and result stream instance
Curvel Aug 21, 2020
13bf97d
Bump to Flutter 1.20.2 stable
Sep 3, 2020
863222f
Support custom "Accept" header
Sep 3, 2020
8e90e94
Refactor plugin a bit & add unit test for multiple subscribers to res…
Sep 3, 2020
4462577
resolve analyzer warnings
Sep 3, 2020
8db09f5
Basic refactor from multiple methods to enqueue an upload to one
Sep 3, 2020
645bc2a
Update README
Sep 3, 2020
70a5e5e
Split UploadWorker into specific raw & form uploader
Sep 4, 2020
601bf5a
Move FlutterEngine management to a separate class
Sep 4, 2020
d04e34b
Move optional WorkManager config to the host application
Sep 4, 2020
557f825
Merge branch '85-accept-header' into azure
Sep 4, 2020
203f7e7
Merge branch 'neohelden/master' into azure
Sep 4, 2020
d39482c
Merge branch 'refactor-api' into azure
Sep 4, 2020
fb01236
Merge branch 'android-refactor' into azure
Sep 4, 2020
24f3219
Azure upload WIP
Sep 4, 2020
e5effec
Read various azure upload config options
Sep 5, 2020
e4f1a06
Fix test
Sep 7, 2020
bd42946
Azure upload worker steps
Sep 7, 2020
71500eb
Implement the clear method
ened Sep 8, 2020
798f821
Various improvements, just a Azure SDK warning remaining
ened Sep 8, 2020
d4cf551
Conditional logging & container creation
Sep 9, 2020
4bd38f0
Merge branch 'master' of https://github.com/BlueChilli/flutter_upload…
Sep 10, 2020
657367d
Disable retries in Azure
Sep 10, 2020
7ee07a2
Reformat Android
Sep 10, 2020
4c65d0f
Import the Azure SDK for iOS
Sep 10, 2020
80d2b59
iOS work
Sep 11, 2020
4c96b14
Support blockSize
Sep 17, 2020
7b41a9b
Do not touch container when createContainer is false
Sep 21, 2020
6f68c27
various isolate improvements
Sep 21, 2020
e530b5e
Fix a swift5 compatibility issue
ened Sep 21, 2020
558642b
More fixes on completion & queueing
Sep 22, 2020
7500760
Merge branch 'azure' of github.com:ened/flutter_uploader into azure
Sep 22, 2020
c91a267
flutter_local_notifications use actions branch for example
Dec 18, 2020
5bb907b
Merge branch 'master' into azure
Dec 18, 2020
4535065
Cleanups
Dec 18, 2020
7dc994b
Resolve merge errors
Dec 18, 2020
d5abe0e
Merge branch 'master' into azure
Dec 22, 2020
63d0d11
Merge branch 'master' into azure
Dec 22, 2020
a5c4599
ensure to free UploadObserver objects quickly
Dec 23, 2020
9344dd7
formatting
ened Dec 26, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .fvm/flutter_sdk
3 changes: 3 additions & 0 deletions .fvm/fvm_config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"flutterSdkVersion": "1.22.5"
}
44 changes: 24 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,29 +70,33 @@ func registerPlugins(registry: FlutterPluginRegistry) {

### Optional configuration:

- **Configure maximum number of concurrent tasks:** the plugin depends on `WorkManager` library and `WorkManager` depends on the number of available processor to configure the maximum number of tasks running at a moment. You can setup a fixed number for this configuration by adding following codes to your `AndroidManifest.xml`:
#### Configure maximum number of concurrent tasks

```xml
<provider
android:name="androidx.work.impl.WorkManagerInitializer"
android:authorities="${applicationId}.workmanager-init"
android:enabled="false"
android:exported="false" />

<provider
android:name="com.bluechilli.flutteruploader.FlutterUploaderInitializer"
android:authorities="${applicationId}.flutter-upload-init"
android:exported="false">
<!-- changes this number to configure the maximum number of concurrent tasks -->
<meta-data
android:name="com.bluechilli.flutterupload.MAX_CONCURRENT_TASKS"
android:value="3" />

<!-- changes this number to configure connection timeout for the upload http request -->
<meta-data android:name="com.bluechilli.flutteruploader.UPLOAD_CONNECTION_TIMEOUT_IN_SECONDS" android:value="3600" />
</provider>
The plugin depends on the `WorkManager` library. The configuration can be done using the instructions at [https://developer.android.com/topic/libraries/architecture/workmanager/advanced/custom-configuration](https://developer.android.com/topic/libraries/architecture/workmanager/advanced/custom-configuration).

The example project shows a custom configuration of up to 10 simultaneous uploads.

Two steps are required:

Depend on the appropriate work-runtime in your host App.
``` gradle
implementation "androidx.work:work-runtime:$work_version"
```

Override the default `Application` and implement the `androidx.work.Configuration.Provider` interface:

``` java
@NonNull
@Override
public Configuration getWorkManagerConfiguration() {
return new Configuration.Builder()
.setMinimumLoggingLevel(android.util.Log.INFO)
.setExecutor(Executors.newFixedThreadPool(10))
.build();
}
```


## Usage

#### Import package:
Expand Down
2 changes: 2 additions & 0 deletions android/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ android {
}

dependencies {
implementation 'com.microsoft.azure.android:azure-storage-android:2.0.0@aar'

implementation "androidx.work:work-runtime:2.4.0"
implementation "androidx.concurrent:concurrent-futures:1.1.0"
implementation "androidx.annotation:annotation:1.1.0"
Expand Down
5 changes: 4 additions & 1 deletion android/src/main/AndroidManifest.xml
Original file line number Diff line number Diff line change
@@ -1,2 +1,5 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.bluechilli.flutteruploader">
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.bluechilli.flutteruploader">

<uses-permission android:name="android.permission.INTERNET" />
</manifest>
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
package com.bluechilli.flutteruploader;

import static com.bluechilli.flutteruploader.UploadWorker.EXTRA_ID;
import static com.bluechilli.flutteruploader.UploadWorker.EXTRA_STATUS;
import static com.bluechilli.flutteruploader.UploadWorker.EXTRA_STATUS_CODE;

import android.content.Context;
import android.content.SharedPreferences;
import android.util.Log;
import androidx.annotation.NonNull;
import androidx.concurrent.futures.CallbackToFutureAdapter;
import androidx.work.Data;
import androidx.work.ListenableWorker;
import androidx.work.WorkerParameters;
import com.google.common.util.concurrent.ListenableFuture;
import com.microsoft.azure.storage.AccessCondition;
import com.microsoft.azure.storage.CloudStorageAccount;
import com.microsoft.azure.storage.OperationContext;
import com.microsoft.azure.storage.RetryNoRetry;
import com.microsoft.azure.storage.blob.BlobRequestOptions;
import com.microsoft.azure.storage.blob.CloudAppendBlob;
import com.microsoft.azure.storage.blob.CloudBlobClient;
import com.microsoft.azure.storage.blob.CloudBlobContainer;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.io.RandomAccessFile;
import java.nio.channels.Channels;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;

public class AzureUploadWorker extends ListenableWorker {

private static final String TAG = "AzureUploadWorker";

/**
* @param appContext The application {@link Context}
* @param workerParams Parameters to setup the internal state of this worker
*/
public AzureUploadWorker(@NonNull Context appContext, @NonNull WorkerParameters workerParams) {
super(appContext, workerParams);
}

private Executor backgroundExecutor = Executors.newSingleThreadExecutor();

@NonNull
@Override
public ListenableFuture<Result> startWork() {
FlutterEngineHelper.start(getApplicationContext());

return CallbackToFutureAdapter.getFuture(
completer -> {
backgroundExecutor.execute(
() -> {
try {
final Result result = doWorkInternal();
completer.set(result);
} catch (Throwable e) {
Log.e(TAG, "Error while uploading to Azure", e);
completer.setException(e);
} finally {
// Do not destroy the engine at this very moment.
// Keep it running in the background for just a little while.
// stopEngine();
}
});

return getId().toString();
});
}

private Result doWorkInternal() throws Throwable {
final String connectionString = getInputData().getString("connectionString");
final String containerName = getInputData().getString("container");
final boolean createContainer = getInputData().getBoolean("createContainer", false);
final String blobName = getInputData().getString("blobName");
final String path = getInputData().getString("path");
final int blockSize = getInputData().getInt("blockSize", 1024 * 1024);

final SharedPreferences preferences =
getApplicationContext().getSharedPreferences("AzureUploadWorker", Context.MODE_PRIVATE);
final String bytesWrittenKey = "bytesWritten." + getId();

// Log.d(TAG, "bytesWrittenKey: " + bytesWrittenKey);

long bytesWritten = preferences.getInt(bytesWrittenKey, 0);

// Log.d(TAG, "bytesWritten : " + bytesWritten);

CloudStorageAccount account = CloudStorageAccount.parse(connectionString);
CloudBlobClient blobClient = account.createCloudBlobClient();

final CloudBlobContainer container = blobClient.getContainerReference(containerName);

final OperationContext opContext = new OperationContext();
// opContext.setLogLevel(BuildConfig.DEBUG ? Log.VERBOSE : Log.WARN);
opContext.setLogLevel(Log.WARN);

final BlobRequestOptions options = new BlobRequestOptions();
options.setRetryPolicyFactory(new RetryNoRetry());

if (createContainer) {
container.createIfNotExists(options, opContext);
}

final CloudAppendBlob appendBlob = container.getAppendBlobReference(blobName);

if (bytesWritten == 0) {
appendBlob.createOrReplace(AccessCondition.generateEmptyCondition(), options, opContext);
}

try (final RandomAccessFile file = new RandomAccessFile(path, "r");
final InputStream is = Channels.newInputStream(file.getChannel())) {
final long contentLength = file.length();

Log.d(TAG, "file contentLength: " + contentLength + ", blockSize: " + blockSize);
if (bytesWritten != 0) {
if (is.skip(bytesWritten) != bytesWritten) {
throw new IllegalArgumentException("source file length mismatch?");
}
}

while (bytesWritten < contentLength && !isStopped()) {
final long thisBlock = Math.min(contentLength - bytesWritten, blockSize);

Log.d(TAG, "Appending block at " + bytesWritten + ", thisBlock: " + thisBlock);

appendBlob.append(
is, thisBlock, AccessCondition.generateEmptyCondition(), options, opContext);

bytesWritten += thisBlock;

double p = ((double) (bytesWritten + thisBlock) / (double) contentLength) * 100;
int progress = (int) Math.round(p);

if (!isStopped()) {
setProgressAsync(
new Data.Builder()
.putInt("status", UploadStatus.RUNNING)
.putInt("progress", progress)
.build());
}

preferences.edit().putInt(bytesWrittenKey, (int) bytesWritten).apply();
}
} catch (FileNotFoundException e) {
Log.e(TAG, "Source path not found: " + path, e);
preferences.edit().remove(bytesWrittenKey).apply();
return Result.failure();
} catch (IOException e) {
return Result.retry();
} catch (Exception e) {
Log.e(TAG, "Unrecoverable exception: " + e);
preferences.edit().remove(bytesWrittenKey).apply();
return Result.failure();
}

final Data.Builder output =
new Data.Builder()
.putString(EXTRA_ID, getId().toString())
.putInt(EXTRA_STATUS, UploadStatus.COMPLETE)
.putInt(EXTRA_STATUS_CODE, 200);

return Result.success(output.build());
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
package com.bluechilli.flutteruploader;

import android.content.Context;
import androidx.annotation.Nullable;
import io.flutter.embedding.engine.FlutterEngine;
import io.flutter.embedding.engine.dart.DartExecutor;
import io.flutter.view.FlutterCallbackInformation;
import io.flutter.view.FlutterMain;

public class FlutterEngineHelper {
@Nullable private static FlutterEngine engine;

public static void start(Context context) {
long callbackHandle = SharedPreferenceHelper.getCallbackHandle(context);

if (callbackHandle != -1L && engine == null) {
engine = new FlutterEngine(context);
FlutterMain.ensureInitializationComplete(context, null);

FlutterCallbackInformation callbackInfo =
FlutterCallbackInformation.lookupCallbackInformation(callbackHandle);
String dartBundlePath = FlutterMain.findAppBundlePath();

engine
.getDartExecutor()
.executeDartCallback(
new DartExecutor.DartCallback(context.getAssets(), dartBundlePath, callbackInfo));
}
}

// private void stopEngine() {
// Log.d(TAG, "Destroying worker engine.");
//
// if (engine != null) {
// try {
// engine.destroy();
// } catch (Throwable e) {
// Log.e(TAG, "Can not destroy engine", e);
// }
// engine = null;
// }
// }
}
Loading