Welcome to Dash Crush, your walk-through tutorial to advanced interactive Data Viz dashboards using Dash Plotly.
In the previous parts, we learned how to save the state of your app to a JSON file.

In this chapter, we will further explore the topic with a complementary part: deserialization.

TL;DR

In this chapter:

1. The first usage of modals
2. Uploading files from a local machine
3. Restoring the state of the app from the config
4. The first use of DataTable

Introduction

In this chapter, we will learn deserialization of state of your app from JSON files.
If your dashboard grows more and more complex and its users need to e.g. run repetitive tasks using similar settings but on different datasets, it may be beneficial for them to set all the controls once and export the state of the app to a JSON file. Next time they will only load the config file from their machine instead of setting all the controls manually.

As I already mentioned, Dash Enterprise provides paid functionality of snapshots. (https://plotly.com/dash/snapshot-engine/).

The code

We will reuse the code from the previous chapter and add a deserialization mechanism on top of it. We will create a "Load config" button and modal and add some callbacks to apply the loaded configuration to the dashboard.

Install packages

Create and activate your virtual environment and install the following packages:

pip install dash pandas dash-bootstrap-components colorlover

Layout

Our layout remains mostly unchanged. We will add only the "Load config" button to the navbar. For a nicer appearance, we also place a spacer above the navbar. In the same file, we create the config import modal. The resulting navbar will be ('src/layout/navbar.py').

# src/layout/navbar.py

import dash_bootstrap_components as dbc
from dash import html
from src.layout.grid import spacer
from src.layout.style import NAVBAR_DISPLAY_STYLE

def navbar():
    """Navbar"""

    return html.Div(
        html.Nav(
            children=[
                spacer(10),
                dbc.Row(
                    [
                        dbc.Col(
                            html.Label(
                                "Welcome to nanostructure visualization dashboard",
                                style=NAVBAR_DISPLAY_STYLE,
                                className="text-primary",
                            ),
                            width={"size": 6, "offset": 3},
                        ),
                        dbc.Col(
                            [
                                dbc.Button(
                                    [html.I(className="fi-upload"), " Load config"],
                                    id="load_config_button",
                                    color="primary",
                                    outline=True,
                                    style={"width": "100%"},
                                ),
                                config_import_modal(),
                            ],
                            width={"size": 2, "offset": 1},
                        ),
                    ],
                ),
            ]
        )
    )

The addition of spacer changes the total height of the navbar. Therefore, we will slightly adjust the CSS assets/dc-graph.css

/* assets/dc-graph.css`

.dc-graph {
   height: calc(100vh - 60px); /* full-visible-height - navbar  */
}

Moreover, we will make a change to NAVBAR_DISPLAY_STYLE in src/layout/style.py by removing the line:

    "paddingTop": "6px",

The style changes are only cosmetic.

Config import modal

To restore the state of our app from a file, we need to upload the file first. Luckily, the Dash ecosystem provides all the components needed. We will place the components on a pop-up screen called Modal (provided by dbc). Our code for the config import modal will be located in src/layout/config_import.py. The model consists of:

• dbc.ModalHeader containing title
• dbc.ModalBody containing main component:
  • dcc.Upload for actual file upload
  • dash_table for listing uploaded file(s)
• dbc.ModalFooter with navigation buttons Load and Cancel

Here is the code:

# src/layout/config_import.py

from dash import dash_table, dcc, html
import dash_bootstrap_components as dbc
from src.layout.grid import spacer
from src.layout import style

def config_import_modal():
    """Modal for config imports"""
    modal = dbc.Modal(
        [
            dbc.ModalHeader("Import JSON configuration file"),
            dbc.ModalBody(upload_pane()),
            import_modal_footer(),
        ],
        id="config_import_modal",
        size="xl",
        backdrop="static",
    )
    return modal

Modal footer

For better readability, we wrap the modal footer in a helper method:

def import_modal_footer():
    """footer of import modal"""
    return dbc.ModalFooter(
        [
            dbc.Button("Cancel", id="config_cancel_button", color="primary"),
            dbc.Button("Load", id="config_load_button", color="primary", disabled=True),
        ]
    )

As mentioned above, it is quite simple and contains only two buttons.

Modal body

The modal body contains the main functional components and looks like this:

def upload_pane():
    """Upload pane"""

    return html.Div(
        [
            spacer(10),
            dbc.Row(
                [
                    dcc.Upload(
                        id="config_upload",
                        children=html.P(
                            [
                                "Drag and Drop or ",
                                dbc.Badge("Select File", pill=True, color="primary"),
                            ]
                        ),
                        style=style.UPLOAD_STYLE,
                        multiple=False,
                        accept="application/json",
                    )
                ]
            ),
            dbc.Row([dcc.Loading(upload_table())]),
            dcc.Store(id="config_uploaded_data", data={}),
            dcc.Store(id="loaded_options_store", data=None),
        ]
    )

Let's go through the code step by step:

1. dcc.Upload is a component responsible for selecting and uploading file(s). The parameter multiple defines if one or many files are allowed, and accept works as a filter on displayed MIME-types, making file selection more user-friendly, as they see only files with a given extension.
2. dbc.Badge draws a colored rectangle around its content. I use it for the visual distinction of the Select file functionality.
3. dcc.Loading shows the loading state indication when its content is populated
4. upload_table will be defined below and will be a table info about the loaded file
5. dcc.Store is a temporary storage to preserve the content of the loaded file for further processing.

Uploaded data table

We will display the info about the uploaded data in dash_table. dash_table is one of the methods of tabular data visualization provided by Dash (another is AG grid).

We will define a table with columns Filename and Status and empty content

def upload_table():
    """Table of updated files"""
    columns = [
        {"name": "Filename", "id": "filename", "deletable": False, "renamable": False},
        {
            "name": "Status",
            "id": "status",
            "deletable": False,
            "renamable": False,
        },
    ]
    return dash_table.DataTable(
        id="config_uploaded_files",
        columns=columns,
        data=[],
        style_cell=style.CELL_STYLE,
    )

The DataTable supports a number of parameters for style customization. One of them is cell_style. We will use it to remove cell borders.

Upload style

One last piece of layout is missing: we used UPLOAD_STYLE to change the appearance of the Upload component and CELL_STYLE to customize DashTable. Now, append the following style definition to src/layout/style.py:

# src/upload/style.py

UPLOAD_STYLE = {
    "width": "100%",
    "height": "80px",
    "lineHeight": "60px",
    "borderWidth": "1px",
    "borderStyle": "dashed",
    "borderRadius": "5px",
    "textAlign": "center",
    "marginBottom": "10px",
}

CELL_STYLE = {"border": "none"}

The layout of our app is ready. To bring it to life, we need to add some callbacks.

Callbacks

For interactivity, we need to add callbacks to our app.

Config import modal

The first one will be responsible for handling the opening and closing of the config import modal. It will be defined in src/callbacks/config_import.py

# src/callbacks/config_import.py

from dash import Input, Output, State

def create_callbacks(app):
    """Create config import callbacks"""

    @app.callback(
        Output("config_import_modal", "is_open"),
        Output("config_upload", "contents"),
        Input("load_config_button", "n_clicks"),
        Input("config_cancel_button", "n_clicks"),
        Input("config_load_button", "n_clicks"),
        State("config_import_modal", "is_open"),
        prevent_initial_call=True,
    )
    def _toggle_config_modal(open_click, cancel_clicks, load_clicks, is_open):
        if open_click or cancel_clicks or load_clicks:
            return not is_open, None
        return is_open, None

At this point, if you created the callback in dashboard.py, you would be able to trigger the modal.
Clicking any of the "Load config" / "Cancel" / "Load" buttons changes the modal's is_open property to the opposite. Additionally, we nullify the uploaded content.

Upload config

Next callback handles upload content:

    @app.callback(
        Output("config_load_button", "disabled"),
        Output("config_uploaded_data", "data"),
        Output("config_uploaded_files", "data"),
        Input("config_upload", "contents"),
        State("config_upload", "filename"),
        prevent_initial_call=True,
    )
    def _upload_config(content, filename):
        if not content:
            return True, {}, []

        _, ext = os.path.splitext(filename)

        config = {}
        status = "missing"
        display_name = filename if ext == ".json" else "*.json"
        download_disabled = True

        if ext == ".json":
            _, file_string = content.split(",")

            try:
                with io.BytesIO(base64.b64decode(file_string)) as file:
                    config = json.load(file)
            except (IOError, OSError, json.JSONDecodeError) as err:
                logging.error("Error loading config file: %s", err)
                status = "invalid"
            else:
                version = (
                    config.get("version", "missing") if isinstance(config, dict) else "missing"
                )
                download_disabled = version != JSON_VERSION
                status = "ok" if version == JSON_VERSION else "invalid"

        uploaded_files = [{"filename": display_name, "status": status}]

        return download_disabled, config, uploaded_files

Let's go through the code step-by-step. Input of _upload_config is base64-encoded file content and filename. We would like them to be a .json file containing a version field. Additionally, for simplicity, we only allow loading files that match JSON_VERSION of our app. We ensure that in few steps:

1. Get file extension with os.path.splitext(filename)
2. Ensure the file type is .json
3. Get the content of the file with _, file_string = content.split(","). The actual file content is located after ,. The part before the comma contains additional information (in my case it was: "data:application/json;base64"), which is not relevant here, so we may ignore it.
4. Decode base64 string: base64.b64decode(file_string)
5. Load the file content to JSON:

               with io.BytesIO(base64.b64decode(file_string)) as file:
                   config = json.load(file)

6. Check if the file version matches the required JSON_VERSION

All the steps handle an error scenario.

Finally, on return we:

• Enable the download button if the config is correct
• Return the decoded content of the uploaded file as a dict
• Populate the uploaded file table with information about the file: its filename and if the version matches

The config is not loaded until we press the "Load" button.

Loading the file

If we loaded a valid config file in the previous step, the "Load" button is not enabled. Let's implement its handling.

    @app.callback(
        Output("loaded_options_store", "data"),
        Input("config_load_button", "n_clicks"),
        State("config_uploaded_data", "data"),
        prevent_initial_call=True,
    )
    def _load_config(_, data):
        """Load new config file"""

        version = data.get("version", None)
        if data.get("version") != JSON_VERSION:
            raise ValueError(
                f"Invalid config file required version: {JSON_VERSION} file version: {version}"
            )

        return data

This callback only copies the config from one store config_uploaded_data to another loaded_options_store. Change of the latter is a trigger for making a dashboard change.

This part could be potentially optimized to reduce the number of intermediate stores. I have implemented it that way, because I usually have some logic to apply to the loaded file on top of the DEFAULT_CONFIG.

Changing the app state:

The next step is applying loaded state to our app. We will use two more for that.

Modifying static components

To modify static components, we need a callback that has all serializable components as outputs (src/layout/deserializer.py):

# src/layout/deserializer.py

from copy import deepcopy
from dash import Input, Output
from dash.exceptions import PreventUpdate

def create_callbacks(app):
    """Create deserialization callbacks"""
    @app.callback(
        Output("bulk_material", "value"),
        Output("bulk_material", "options"),
        Output("json_load_store", "data"),
        Input("loaded_options_store", "data"),
        prevent_initial_call=True,
    )
    def _update_available_options(loaded_config):
        if not loaded_config:
            raise PreventUpdate
        options = loaded_config["options"]
        cards_config = loaded_config["cards"]

        return _format_options(options, cards_config)

def _format_options(options, cards_config):
    formated_options = deepcopy(options)
    formated_options["substrate_options"] = [
        {"label": name, "value": name} for name in options["substrate_options"]
    ]

    # Return cards config to force layer cards refresh
    ret = tuple(formated_options.values()) + (cards_config,)
    return ret

This callback receives the config, modifies the format of stored data to match the one required by various components, and dispatches config values to modify the properties of the requested components. The static components are modified in this place. On the other hand, the config of cards' layers is sent to one more intermediate store loaded_options_store.

Remark: This function requires maintenance every time the structure of config changes!

Modifying dynamic content

As you may remember, previously we created a callback to handle a deck of cards. Now we need to modify it to restore cards using the configuration from loaded_options_store. For that purpose, we need to modify the _handle_deck callback in src/callbacks/deck.py. Add json_load_store as input and call a card restoration routine if this input is the caller. Additionally, to ensure the layer card indexing is monotonically increasing, we will need to do one using the current n_click property of add_card_button.

The beginning of _handle_deck should now look like this:

# src/callbacks/deck.py
from src.layout.card import empty_layer, card_layer
from src.layout.card_body import restore_params

def create_callbacks(app):
    """Create deck callbacks"""

    @app.callback(
        Output("deck", "children"),
        Input("add_card_button", "n_clicks"),
        Input({"role": "card_remove", "index": ALL}, "n_clicks"),
        Input({"role": "card_up", "index": ALL}, "n_clicks"),
        Input({"role": "card_down", "index": ALL}, "n_clicks"),
        Input("json_load_store", "data"),
        State("deck", "children"),
        State("add_card_button", "n_clicks"),
        prevent_initial_call=True,
    )
    def _handle_deck(
        add_clicks,
        remove_clicks,
        up_clicks,
        down_clicks,
        json_config,
        current_cards,
        last_add_clicks,
    ):  # pylint: disable=unused-argument, too-many-arguments
        """Add, remove or move layer card"""
        caller_id = get_caller_id(dash.callback_context)
        out_cards = current_cards

        if caller_id == "add_card_button":
            out_cards.append(empty_layer(add_clicks, MATERIAL_OPTIONS))
        elif caller_id == "json_load_store":
            if json_config is None:
                raise PreventUpdate
            out_cards = restore_cards(last_add_clicks, json_config, MATERIAL_OPTIONS)

_restore_cards function looks like:

def _restore_cards(add_clicks, layers, material_options):
    """restore cards according to cards config"""
    out_cards = []
    # HACK: add cards in a way, so last one has current add_clicks as index
    start_idx = add_clicks - len(layers) + 1
    for card_idx, layer in enumerate(layers):
        index = start_idx + card_idx
        params = restore_params(index, layer["material"], layer["params"])
        card = card_layer(index, material_options, params, layer["material"])
        out_cards.append(card)
    return out_cards

We used a small trick here: we assigned card indices using the current n_click property of add_card_button so that the newest card gets an index equal to n_click. Consequently, using this method, the older card will get a smaller index number, and the next card will get an index equal to n_click+1. As a result, we eliminated the risk of duplicated indices.
For our method to work flawlessly, we need to modify the button definition src/layout/add_pane.py to initialize the n_click with 0 to avoid None values:

                    dbc.Button(
                        [html.I(className="fi-plus"), "Layer"],
                        id="add_card_button",
                        # HACK to prevent None values in `deck:restore_cards`
                        n_clicks=0,
                        color="primary",
                        style={"width": "100%"},
                    )

Next, we need to define the parameter restoration logic. restore_params is defined in src/layout/card_body.py

# src/layout/card_body.py

def restore_params(index, material, values):
    """Fill cards values loaded from JSON"""
    params = deepcopy(dropdowns.MATERIAL_MAPPING[material])
    sections = []
    for name, param_dict in params.items():
        description = {"index": index, "name": name, "material": material}
        curr_value = values.get(name)
        section = _param_section(param_dict, curr_value, description)
        sections.append(section)

    return sections

The difference to the previously defined fill_params accepts values being a dictionary instead of current_value being a list.

Using callback

Finally, we need to create all the callbacks. In dashboard.py modify callback creation with:

# dashboard.py
import src.callbacks.config_import
import src.callbacks.deserializer

and in create_callbacks(app):

    # Append at the end of the function
    src.callbacks.config_import.create_callbacks(app)
    src.callbacks.deserializer.create_callbacks(app)

Starting the app:

Start the app with python dashboard.py. You should see an indication of a running app in your console. The dashboard is available under http://localhost:8050.

Summary:

Quick recap:

1. Use dcc.Upload to load a local file
2. Dispatch loaded config in an opposite way as during serialization
3. Modify the deck callback to handle additional input being JSON config
4. Use dbc.Modal for pop-up
5. dash_table.DataTable is one of the possible ways of handling tabular data

We have used quite a few new components in this chapter. Here are some useful links:

• dbc.Modal: https://dash-bootstrap-components.opensource.faculty.ai/docs/components/modal/
• Upload: https://dash.plotly.com/dash-core-components/upload
• Badge: https://dash-bootstrap-components.opensource.faculty.ai/docs/components/badge/
• Loading: https://dash.plotly.com/dash-core-components/loading
• DataTable: https://dash.plotly.com/datatable
• MIME type list: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types

The complete code from this chapter can be found on GitHub:
https://github.com/ptrhbt/dash-crush/tree/7-serialization

There is also the requirements.txt file which lists all packages used. Just install it to your virtual env with pip install -r requirements.txt.

If you have any questions or comments, let me know.

See you in the next chapter of Dash Crush, where we will start a new topic.