TDM 40200: Project 3 — 2023

Motivation: Dashboards are everywhere — many of our corporate partners' projects are to build dashboards (or dashboard variants)! Dashboards are used to interactively visualize some set of data. Dashboards can be used to display, add, remove, filter, or complete some customized operation to data. Ultimately, a dashboard is really a website focused on displaying data. Dashboards are so popular, there are entire frameworks designed around making them with less effort, faster. Two of the more popular examples of such frameworks are shiny (in R) and dash (in Python). While these tools are incredibly useful, it can be very beneficial to take a step back and build a dashboard (or website) from scratch (we are going to utilize many powerfuly packages and tools that make this far from "scratch", but it will still be more from scratch than those dashboard frameworks).

Context: This is the second in a series of projects focused around slowly building a dashboard. Students will have the opportunity to: create a backend (API) using fastapi, connect the backend to a database using aiosql, use the jinja2 templating engine to create a frontend, use htmx to add "reactivity" to the frontend, create and use forms to insert data into the database, containerize the application so it can be deployed anywhere, and deploy the application to a cloud provider. Each week the project will build on the previous week, however, each week will be self-contained. This means that you can complete the project in any order, and if you miss a week, you can still complete the following project using the provided starting point.

Scope: Python, dashboards

Learning Objectives
  • Create a simple backend server using fastapi.

Make sure to read about, and use the template found here, and the important information about projects submissions here.

Questions

Question 1

Start this project by opening VS Code and logging onto Anvil just like we did in the previous project. Once you have opened a session, make sure to pop open a terminal on Anvil inside VS Code by pressing Cmd+Shift+P (mac) or Ctrl+Shift+P (windows), and typing "Terminal: Create New Terminal".In the terminal, navigate to your $HOME directory, and open a terminal session. In the terminal session, run the following command:

cd

Great. First thing is first. We need to create a new directory, that we will refer to as our "project directory" or "root directory". Call the directory media_app. One of our most complete databases is our imdb database, which you are all likely familiar with. For this reason, it makes the most sense to make our dashboard a media dashboard.

mkdir media_app
cd media_app

Within our media_app directory, we want to organize all of our files and folders. First, create a backend directory. This is where we will keep the source code and critical files that we will use to run our backend. We will use the terms "backend", "webserver", "server", and "api" interchangeably. These are all very common terms for the same thing.

mkdir backend
cd backend

We are going to start off slowly, so we can take our time and understand the critical parts pretty well. Create a new Python mondule called main.py and drop it in your backend directory, and open it up in VS Code.

Drop the following code into main.py, and save it.

from fastapi import FastAPI (1)

app = FastAPI() (2)


@app.get("/") (3)
async def hello(): (4)
    return {"message": "Hello World"}

This is one of the simplest apis you can write with fastapi. Here are some descriptions, line-by-line.

1 We are importing the FastAPI class from the fastapi package. This is the class that we will use to create our api.
2 We are creating an instance of the FastAPI class, called app. This is the object that we will use to run our api, and assign functions to endpoints.
3 We are using the app object to assign a function to the / endpoint. This means that when a user visits the root of our api, they will be served the output of the function. For example, if our api is running at localhost:8000/, then when a user visits localhost:8000/, they will be served the output of the hello function.
4 The hello function. This function is very simple and returns a dict with key "message" and value "Hello World". This is the output that will be served to the user when they visit the root of our api. fastapi will automatically convert the dict to JSON. When displayed in the browser, JSON data will be displayed in a human-readable format, that looks distinctly different than when you’d visit a regular web page like datamine.purdue.edu.

This is all great, but we need to run our api in order to see it in action. To do this, we need to execute our main.py module. In order to run the main.py module, we need to make sure the fastapi package is available in our Python environment, otherwise we will receive an error when we try to import it. Luckily, we already have fastapi installed in our f2022-s2023 environment, we just need to load it up. To load up our f2022-s2023 environment, please run the following in the terminal.

module use /anvil/projects/tdm/opt/core
module load tdm
module load python/f2022-s2023

Before running the commands above, if we were to run which python3 in the terminal, it would point to our system Python on Anvil, which we don’t have any control over — /usr/bin/python3. After running the commands above, running which python3 will point to a shell function which executes our Python code in a singularity container. This version of Python is loaded up with lots of packages, ready for you to use, including fastapi.

Normally, to run a Python module, we would just run a command like:

python3 main.py

However, we are not using the Python interpreter to run our module, we are using a Python ASGI web server called uvicorn. Uvicorn is a very popular web server, and is used by many popular Python web frameworks, including fastapi. To run our module with uvicorn, we need to run the following command.

python3 -m uvicorn main:app --reload

When running a server, you must choose a port to run the server on. A port can have a value from 1 to 65535, however, many of those ports are reserved for certain programs. Each port can only have a program utilizing UDP and TCP protocols on them. Only 1 TCP and 1 UDP per port. By default, fastapi will choose to run your server on port 8000, using the TCP protocol. If someone else happens to be on the same Anvil node as you, and is using port 8000 (a rather popular port number), you will receive an error — that port is already in use.

We’ve created a script to print out an unused port. Run the following command.

find_port
potential output
39937

Now, we can use extra uvicorn arguments to specify the port we want to run out app on.

python3 -m uvicorn main:app --reload --port 39937

Now, we can visit our api in the browser. Visit localhost:39937/ in your browser.

You may have to click "allow" in a VS Code popup asking about forwarding ports. This just makes it so you can go to port 39937 (for example) on your own computer’s browser, and you will essentially be on Anvil’s port 39937.

You should see the following output.

Expected output
Figure 1. Expected output

Great! What does each of the parts of the command mean?

python3 -m uvicorn main:app --reload --port 39937

The python3 -m uvicorn part is just a way to access and run the installed uvicorn app.

The main:app part is telling uvicorn which module and which object to run. In this case, we are telling uvicorn to run the app object in the main.py module. If your current working directory when running the command was in media_app instead of in backend, you would need to run a slightly modified command.

python3 -m uvicorn backend.main:app --reload --port 39937

Here, backend.main translates to in the backend directory in the main.py module. To reiterate, the app object is the object that we are using to run our api, and assign functions to endpoints. For instance, if we modified our code to be the following.

from fastapi import FastAPI

my_app = FastAPI()


@my_app.get("/")
async def hello():
    return {"message": "Hello World"}

Then, we would have to modify our command to be the following.

python3 -m uvicorn backend.main:my_app --reload --port 39937

The --port command is more obvious — it is picking which port we want to run the server on.

Finally, the --reload command is telling uvicorn to reload the server whenever we make a change to our code. This is very useful for development, but should be removed when we are ready to deploy our app. Let’s test it out. While your app is still running, change the key of the returned dict from "message" from "communique". Save the main.py file, and refresh your browser. You should see the following output, and you didn’t even need to restart the server!

Expected output
Figure 2. Expected output

This is useful. This means that you’ll typically just need to run the server and keep it running as you develop your api.

To verify that you mostly understand all of this, please provide the command you would use to run the backend if your current working directory was your $HOME directory. Put your solution in a markdown cell in a Jupyter Notebook.

Items to submit
  • Code used to solve this problem.

  • Output from running the code.

Question 2

In the previous question, we assigned the hello function to the / endpoint. That way, when you are running the server and navigate to localhost:39937/, you will see our "Hello World" message. Let’s add more endpoints to our backend, so we can better understand how this works.

First, instead of accessing our "Hello World" message by going to localhost:39937/, let’s access it by going to localhost:39937/hello. Make the required modification and demonstrate that it works. Submit a screenshot of your browser and the response from the server. The screenshot should include the URL and the response — just like my screenshots in the previous question did. Forget how to include an image in your notebook? See here.

Items to submit
  • Code used to solve this problem.

  • Output from running the code.

Question 3

Fantastic. Let’s add another endpoint. This time, let’s add an endpoint that takes a name and returns a message, just like before, except now instead of "Hello World", the message should be "Hello NAME", where NAME is the name that was passed in. For example, if we passed in the name "drward" to the endpoint, we would expect the following.

Expected output
Figure 3. Expected output

This does an excellent job explaining how to add a path parameter, and what is happening.

Items to submit
  • Code used to solve this problem.

  • Output from running the code.

Question 4

In the previous question, you stripped the path parameter, and passed it to your function. For example, if you had the following code.

@app.get("/some/endpoint/{some_argument}")
async def some_function(some_argument: str):
    return {"output": some_argument}

If you navigated to localhost:39937/some/endpoint/this_is_my_argument, you would see the following output.

output
{"output": "this_is_my_argument"}

The function receives the value from the @app.get("/some/endpoint/{some_argument}") line, and it passes the value, in our example, "this_is_my_argument" to the some_function function. This equates to something like the following.

some_function("this_is_my_argument")

That function simply returns the dict, which is quickly transformed to JSON and returned as the response.

Well, not all values need to be passed from the URL to some function through path parameters. Path parameters are typically used when the information you want to pass through a path parameter has something to do with the structure of the data. For example, our endpoint /hello/NAME doesn’t make a whole lot of sense. Names are not unique, and if we had multiple drwards, we couldn’t access both of their information from the same endpoint. However, if you had something like /users/123/hello, then it would make sense. The 123 could be a unique identifier for a user, and the hello endpoint could return a customized hello message for that specific user.

If you wanted an endpoint to say hello to any old person — not necessarily to a individual in your database, for instance, then there is another way that makes lots more sense that using a custom endpoint like /hello/NAME.

Instead, you can use a query parameter. A query parameter is a parameter that is passed through the URL, but is not part of the path. This does an excellent job explaining what a query parameter is, and how to use them in fastapi.

Update your /hello/ endpoint to accept a query parameter called name. The endpoint should still return the same message, but this time it should use the query parameter instead of the path parameter. Demonstrate that it works by submitting a screenshot of your browser and the response from the server. The screenshot should include the URL and the response. Below is an example, passing "drward" — please choose a different name for your example.

Expected output
Figure 4. Expected output
Items to submit
  • Code used to solve this problem.

  • Output from running the code.

Question 5

So, you’ve typed 'http://localhost:39937/hello?name=drward' into your browser, and you’ve seen the message "Hello drward". That’s great, but it is time to define some concepts. When you type that URL into your browser and hit enter, what is happening? Your browser makes a GET request. GET is one of the HTTP methods. GET is used to retrieve information from a server. In this case, we are retrieving information from our server. This information just happens to be some JSON with a "Hello World" message.

The "Hello World" message is part of the response. The response is the information that is returned from the server. A response has three primary components: a status line, one or more headers, and a body.

Use your /hello?name=person endpoint. At the top of your browser, you should see "JSON" (what you are used to seeing), "Raw Data" and "Headers". In a markdown cell, copy and paste the "Raw Data" (your response body), and the "Headers" (your response headers).

Next, modify your hello endpoint to return an additional response header with key "attitude" and value "sassy". In addition, change the status code from 200 to 501. Demonstrate that it works by submitting screenshots of your browser and the response from the server. The screenshot should include the URL and the response. Below is an example, passing "drward" — please choose a different name for your example.

Dig around in the offical docs to figure out how to add a response header, and how to change the status code.

In order to see the status code from the browser, you will need to open the Inspector and click on "Console". You may need to make the request again (refresh the web page) in order to see the status code.

Response body example
Figure 5. Response body example
Response headers example
Figure 6. Response headers example
Status code example
Figure 7. Status code example
Items to submit
  • Code used to solve this problem.

  • Output from running the code.

Please make sure to double check that your submission is complete, and contains all of your code and output before submitting. If you are on a spotty internet connection, it is recommended to download your submission after submitting it to make sure what you think you submitted, was what you actually submitted.

In addition, please review our submission guidelines before submitting your project.