Skip to content

Working with Attributes

In the metadata schema, every object has attributes that are represented by simple types like strings and numbers. For example,News objects have a source and Publication objects have a title. This page details how to change attributes on existing metadata types.

Here is an example of the code that defines the start_date attribute of the Event class: 

class EventBase(AIResourceBase):
    start_date: datetime | None = Field(
        description="The start date and time of the event, formatted using the ISO 8601 date-time "
        "format.",
        default=None,
        schema_extra={"example": "2021-02-03T15:15:00"},
    )
Let's unpack this statement. The start_date: datetime defines the name of the attribute (start_date) and the type of the attribute (datetime | None, an optional datetime object). Python type hints are used by Pydantic to do input validation, and by SQLAlchemy to infer column types in the database - this all happens "under the hood" by SQLModel. The Field object allows to define additional information about the attribute:

  • The content of description will be shown on the generated documentation pages, and should clear up any ambiguity on how to interpret the attribute.
  • The default parameter may be specified to provide a default value that will be used for the attribute if no explicit value is provided.
  • The schema_extra parameter may specify additional information, the most important of which is example. The example value will be used in the formatting of example responses generated by the Swagger and ReDoc pages.

Adding an attribute to an existing asset type

This guide explains how to add a simple attribute, i.e., an attribute which does not refer to any other table, to an existing asset type.

Adding an attribute to an asset requires three things:

  • An update to the schema in Python, i.e., a change to the SQLModel object.
  • Added or updated tests that reflect the change.
  • The addition of a migration script, which specifies how the database schema should be updated.

This guide will discuss them in order, using the example of adding a source attribute to the News asset (see also pull request).

Updating the schema in Python

Navigate to the asset definition you want to change. For simple attributes this is the class that ends in "Base", e.g., "NewsBase". There, add the attribute and define its metadata through adding type hints and setting it to a SQLModel Field. There are already examples in the code base of many different types of constraints (e.g., string minimum or maximum lengths, specifying defaults, and so on).

Working with Strings

In Python there is no limit to the amount of characters in a string. However, when working with a database it may be wise to put constraints on the length of the string. This is more efficient and may in some cases lead to avoiding unwanted mistakes. When picking a string length, please choose from the pre-existing options in src.database.model.field_length.py. If in doubt, go for a larger size.

Example: event name

Assume that we wanted to add a "name" field to Event to store its name. We might consider a few examples names, such as "Hacktoberfest" or "Forty-Second International Conference on Machine Learning". Both fall under 64 characters (SHORT), but the latter is already cutting it close. In that case, it's probably smart to go one bigger: 256 characters (NORMAL).

from database.model.field_length import NORMAL

class Event(...):
    name: str = Field(max_length=NORMAL, schema_extra={"example": "The name of this event."})

Adding or updating tests that reflect the change

Most of the time, it should be sufficient to navigate to the tests of the resource router for the resource you are trying to edit. In many cases, there is only one such test in a file named test_router_ASSET_TYPE.py where ASSET_TYPE is e.g., "news", and the one test is called "test_happy_path". Here, you may add or update a line that tests setting the specific attribute that is added.

Adding a migration script

We start off by having Alembic generate a new migration script. Follow the "Using Alembic" instructions to generate a template migration script. In this script, you will find two empty functions:

def upgrade() -> None:
    pass


def downgrade() -> None:
    pass

All you have to do to finish the migration script is implement these two functions. The upgrade function specifies the instruction to go from the old database schema (without the added attribute) to the new one (with the added attribute). Below, you'll find some examples.

def upgrade() -> None:
    op.add_column(
        table_name="news",
        column=Column("source", String(LONG), nullable=True),
    )


def downgrade() -> None:
    op.drop_column(table_name="news", column_name="source")

def upgrade() -> None:
    op.add_column(
        table_name="event",
        column=Column("max_participants", int, nullable=False),
    )


def downgrade() -> None:
    op.drop_column(table_name="event", column_name="max_participants")

Pay attention to: - Specifying whether or not the column is nullable. sqlalchemy.Columns are nullable by default. We strongly encourage you to make this explicit. - The name of the column. Getting the name of the table wrong should result in an error during migration, but getting the name of the column wrong will simply lead to unexpected errors in the REST API. - Any other constraints, such as the maximum length of a string.

Note that downgrading procedures may lead to a loss of data. This is unavoidable.