Asynchronous Metrics Download in Admin Panel

[saifrk]

Implementation of the Download Metrics Button in Django Admin:

1. Button addition:
   • Added a 'metrics' button using changelist_view method
2. Task generation:
   • Triggered by button click, the download_metrics:
     • Generates a unique task_id (UUID).
     • Initiates an asynchronous Celery task generate_metrics.delay().
     • Displays a download link message
     • Redirects back to the collection list page.
3. Cleanup mechanism:
   • At the start of generate_metrics task:
     • Scans for files named "metrics_*.csv" or "metrics.tmp" in the metrics directory.
     • Removes old files, keeping only those related to the current task_id.
4. Background processing:
   • Asynchronous generate_metrics task:
     • Gathers metrics data about collections.
     • Creates a CSV file named metrics_{task_id}.csv in MEDIA_ROOT/metrics/.
     • Uses a .tmp file to indicate ongoing processing.
5. File retrieval:
   • When downloading via the link:
     • get_metrics_file checks if the file exists and is substantial (>100 bytes).
     • Serves the file as a downloadable CSV if ready.
     • If still processing, displays status messages and offers retry option

Note: Currently there are 3 metrics that the script calculates, for all the 3 to run and a CSV to be written and downloaded, it takes over 15 mins. Thus it was necessary to have it run asynchronously for better user exp.