Skip to content

Commit e4b4853

Browse files
timurbazhirovtimurbazhirov
authored
Merge pull request #321 from Exabyte-io/docs/SOF-7580
SOF-7580: video tutorial on using jupyter notebook for data analysis
2 parents 9e6b61d + 58c0c73 commit e4b4853

File tree

4 files changed

+196
-4
lines changed

4 files changed

+196
-4
lines changed

‎lang/en/docs/jupyterlite/accessing-jupyterlite.md

+10
Original file line numberDiff line numberDiff line change
@@ -20,3 +20,13 @@ To access [JupyterLite](https://jupyterlite.mat3ra.com/lab/index.html) directly,
2020
```
2121
https://jupyterlite.mat3ra.com/lab/index.html
2222
```
23+
24+
25+
## JupyterLite for data analysis
26+
27+
In the below tutorial, we present how we can use JupyterLite session in Mat3ra
28+
platform to postprocess or analyze data.
29+
30+
<div class="video-wrapper">
31+
<iframe class="gifffer" width="100%" height="100%" src="https://www.youtube.com/embed/PXosTghiAzs" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
32+
</div>

‎lang/en/docs/jupyterlite/jupyter-lite.json

+182
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,182 @@
1+
{
2+
"descriptionLinks": [
3+
"Using Jupyter notebooks for data analysis: https://docs.mat3ra.com/jupyterlite/accessing-jupyterlite/"
4+
],
5+
"description": "We present how we can use Jupyter notebooks in Mat3ra platform for data analysis.",
6+
"tags": [
7+
{
8+
"...": "../metadata/general.json#/tags"
9+
},
10+
"Jupyter",
11+
"Python"
12+
],
13+
"title": "Mat3ra Tutorial: Using Jupyter notebooks for data analysis",
14+
"youTubeCaptions": [
15+
{
16+
"text": "Hi, <break time='0.5'/> in this short tutorial, we are going to present how we can use jupyter notebooks in matera platform for data analysis.",
17+
"startTime": "00:00:00.500",
18+
"endTime": "00:00:08.000"
19+
},
20+
{
21+
"text": "It's a great option for those who are familiar with Python.",
22+
"startTime": "00:00:08.500",
23+
"endTime": "00:00:12.000"
24+
},
25+
{
26+
"text": "Here we will specifically focus on how it can help us simplify the process of analyzing data obtained from DFT simulation..",
27+
"startTime": "00:00:13.000",
28+
"endTime": "00:00:22.000"
29+
},
30+
{
31+
"text": "Let's head over to our platform. Use a web browser and visit platform dot matera dot com.",
32+
"startTime": "00:00:23.000",
33+
"endTime": "00:00:29.000"
34+
},
35+
{
36+
"text": "We are going to create a DFT workflow to calculate bandstructure of silicon using quantum espresso.",
37+
"startTime": "00:00:30.000",
38+
"endTime": "00:00:36.000"
39+
},
40+
{
41+
"text": "This calculation has three steps, <break time='1.0'/> first we perform self consistent field calculation.",
42+
"startTime": "00:00:37.000",
43+
"endTime": "00:00:43.000"
44+
},
45+
{
46+
"text": "Then we add a unit for bands calculation.",
47+
"startTime": "00:00:44.000",
48+
"endTime": "00:00:47.000"
49+
},
50+
{
51+
"text": "Finally, we add unit for postprocessing of bands using bands dot X.",
52+
"startTime": "00:00:52.000",
53+
"endTime": "00:00:55.000"
54+
},
55+
{
56+
"text": "Save and exit workflow. <break time='0.5'/> Create a job with the workflow we have just created.",
57+
"startTime": "00:00:58.000",
58+
"endTime": "00:01:04.000"
59+
},
60+
{
61+
"text": "Submit job for execution.",
62+
"startTime": "00:01:20.000",
63+
"endTime": "00:01:21.000"
64+
},
65+
{
66+
"text": "Once the job is finished, we can see the summary of various results.",
67+
"startTime": "00:01:25.000",
68+
"endTime": "00:01:29.000"
69+
},
70+
{
71+
"text": "Navigate to the files tab, and we will see all the output files are listed here.",
72+
"startTime": "00:01:34.000",
73+
"endTime": "00:01:39.000"
74+
},
75+
{
76+
"text": "Now, we would like to do the post-processing of these output files using Python Jupyter notebook.",
77+
"startTime": "00:01:40.000",
78+
"endTime": "00:01:45.000"
79+
},
80+
{
81+
"text": "There are a couple of different ways to run Jupyter notebook in our platform.",
82+
"startTime": "00:01:46.000",
83+
"endTime": "00:01:50.000"
84+
},
85+
{
86+
"text": "Perhaps, the quickest way to launch Jupyter lite. Click the console icon on the top right and select Jupyter lite.",
87+
"startTime": "00:01:51.000",
88+
"endTime": "00:01:57.000"
89+
},
90+
{
91+
"text": "Let's create a new notebook, <break time='2.0'/> and rename the notebook to bands analysis.",
92+
"startTime": "00:02:01.000",
93+
"endTime": "00:02:06.000"
94+
},
95+
{
96+
"text": "Let's also open an example notebook.",
97+
"startTime": "00:02:10.000",
98+
"endTime": "00:02:13.000"
99+
},
100+
{
101+
"text": "Here we will see that it is entirely possible to perform all the steps from a Jupyter notebook.",
102+
"startTime": "00:02:16.000",
103+
"endTime": "00:02:22.000"
104+
},
105+
{
106+
"text": "Including authentication, <break time='0.25'/> workflow creation, <break time='0.25'/> job submission, <break time='0.25'/> job monitoring and <break time='0.25'/> fetch results.",
107+
"startTime": "00:02:22.500",
108+
"endTime": "00:02:29.000"
109+
},
110+
{
111+
"text": "Now, let's copy the authentication part and run inside our bands analysis notebook.",
112+
"startTime": "00:02:30.000",
113+
"endTime": "00:02:35.000"
114+
},
115+
{
116+
"text": "It will generate API keys and authenticate user seamlessly.",
117+
"startTime": "00:02:36.000",
118+
"endTime": "00:02:39.000"
119+
},
120+
{
121+
"text": "It will also install a list of default packages.",
122+
"startTime": "00:02:40.000",
123+
"endTime": "00:02:43.000"
124+
},
125+
{
126+
"text": "Next we initialize the jobs endpoint.",
127+
"startTime": "00:02:45.000",
128+
"endTime": "00:02:47.000"
129+
},
130+
{
131+
"text": "Then we would like to fetch the output file.",
132+
"startTime": "00:02:48.000",
133+
"endTime": "00:02:50.000"
134+
},
135+
{
136+
"text": "For this we need the job ID.",
137+
"startTime": "00:02:55.000",
138+
"endTime": "00:02:57.000"
139+
},
140+
{
141+
"text": "Let's go back to job page.",
142+
"startTime": "00:02:58.000",
143+
"endTime": "00:03:00.000"
144+
},
145+
{
146+
"text": "And copy the job ID.",
147+
"startTime": "00:03:02.000",
148+
"endTime": "00:03:04.000"
149+
},
150+
{
151+
"text": "We would like to fetch the bands dot dat dot GNU file for bandstructure analysis.",
152+
"startTime": "00:03:06.000",
153+
"endTime": "00:03:10.000"
154+
},
155+
{
156+
"text": "Let's save the results in a file named data dot TXT.",
157+
"startTime": "00:03:12.000",
158+
"endTime": "00:03:15.000"
159+
},
160+
{
161+
"text": "Finally, we can use matplotlib to make our bandstructure plot.",
162+
"startTime": "00:03:17.000",
163+
"endTime": "00:03:21.000"
164+
},
165+
{
166+
"text": "This use case is not specific to DFT data analysis. You can use Python notebooks to analyze or postprocess any data in our platform.",
167+
"startTime": "00:03:22.000",
168+
"endTime": "00:03:31.000"
169+
},
170+
{
171+
"text": "Now, I hope you are excited to visit platform dot matera dot com, and give it a try.",
172+
"startTime": "00:03:32.000",
173+
"endTime": "00:03:36.000"
174+
},
175+
{
176+
"text": "Thank you for following this tutorial and using our platform.",
177+
"startTime": "00:03:37.000",
178+
"endTime": "00:03:41.000"
179+
}
180+
],
181+
"youTubeId": "PXosTghiAzs"
182+
}

‎lang/en/docs/tutorials/other/jupyter.md

+3-3
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,11 @@ This tutorial page explains how to create a Jupyter Notebook environment through
44

55
## Generate RESTFul API Tokens
66

7-
The Jupyter notebook environment in the present tutorial is used to run an IPython notebook from [Exabyte API Examples Repository](../../rest-api/api-examples.md) in which a connection is made to the RESTFul API to retrieve a list of materials. In order to establish the connection, one should generate RESTFul API tokens following the steps described in [here](../../rest-api/authentication.md).
7+
The Jupyter notebook environment in the present tutorial is used to run an IPython notebook from [Exabyte API Examples Repository](../../rest-api/api-examples.md) in which a connection is made to the RESTFul API to retrieve a list of materials. In order to establish the connection, one should generate RESTFul API tokens following the steps described in [here](../../rest-api/authentication.md).
88

99
## Upload IPython Notebooks
1010

11-
Jupyter Notebook is started on the account [Dropbox](../../data-in-objectstorage/dropbox.md) directory. This directory provides users with access to previously uploaded/created IPython notebooks. Here, **settings.py** file contains the variables required to configure the RESTFul API endpoints and **get_materials_by_formula.ipynb** from the [Exabyte API Examples Github Repository](../../rest-api/api-examples.md) are uploaded to Dropbox to be later used inside the Jupyter notebook environment.
11+
Jupyter Notebook is started on the account [Dropbox](../../data-in-objectstorage/dropbox.md) directory. This directory provides users with access to previously uploaded/created IPython notebooks. Here, **settings.py** file contains the variables required to configure the RESTFul API endpoints and **get_materials_by_formula.ipynb** from the [Exabyte API Examples Github Repository](../../rest-api/api-examples.md) are uploaded to Dropbox to be later used inside the Jupyter notebook environment.
1212

1313
## Create Jupyter Job
1414

@@ -20,7 +20,7 @@ Jupyter Notebook installation and configuration is handled through the Jupyter N
2020

2121
## Adjust Jupyter Notebook Environment
2222

23-
Jupyter Notebook is installed inside a Python [virtual environment](https://virtualenv.pypa.io/en/latest/) with no additional packages initially. The environment can be customized by navigating to the [workflow tab](../../jobs-designer/workflow-tab.md) and adjusting the **configure.sh** script located inside the **notebook** unit. Here, we install [Exabyte API Client](../../rest-api/api-client.md) Python package to connect to Exabyte RESTFul API.
23+
Jupyter Notebook is installed inside a Python [virtual environment](https://virtualenv.pypa.io/en/latest/) with no additional packages initially. The environment can be customized by navigating to the [workflow tab](../../jobs-designer/workflow-tab.md) and adjusting the **configure.sh** script located inside the **notebook** unit. Here, we install [Exabyte API Client](../../rest-api/api-client.md) Python package to connect to Exabyte RESTFul API.
2424

2525
## Submit Job
2626

‎requirements.txt

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
babel==2.16.0
22
backports-abc==0.5
3-
cachetools==5.5.0
3+
cachetools==5.5.2
44
certifi==2024.8.30
55
chardet==4.0.0
66
charset-normalizer==3.4.1

0 commit comments

Comments
 (0)