Extending Schemas

Pre-processing and Post-processing Methods

Data pre-processing and post-processing methods can be registered using the pre_load, post_load, pre_dump, and post_dump decorators.

from marshmallow import Schema, fields, pre_load

class UserSchema(Schema):
    name = fields.Str()
    slug = fields.Str()

    @pre_load
    def slugify_name(self, in_data):
        in_data['slug'] = in_data['slug'].lower().strip().replace(' ', '-')
        return in_data

schema = UserSchema()
result, errors = schema.load({'name': 'Steve', 'slug': 'Steve Loria '})
result['slug']  # => 'steve-loria'

Passing “many”

By default, pre- and post-processing methods receive one object/datum at a time, transparently handling the many parameter passed to the schema at runtime.

In cases where your pre- and post-processing methods need to receive the input collection when many=True, add pass_many=True to the method decorators. The method will receive the input data (which may be a single datum or a collection) and the boolean value of many.

Example: Enveloping

One common use case is to wrap data in a namespace upon serialization and unwrap the data during deserialization.

from marshmallow import Schema, fields, pre_load, post_load, post_dump

class BaseSchema(Schema):
    # Custom options
    __envelope__ = {
        'single': None,
        'many': None
    }
    __model__ = User

    def get_envelope_key(self, many):
        """Helper to get the envelope key."""
        key = self.__envelope__['many'] if many else self.__envelope__['single']
        assert key is not None, "Envelope key undefined"
        return key

    @pre_load(pass_many=True)
    def unwrap_envelope(self, data, many):
        key = self.get_envelope_key(many)
        return data[key]

    @post_dump(pass_many=True)
    def wrap_with_envelope(self, data, many):
        key = self.get_envelope_key(many)
        return {key: data}

    @post_load
    def make_object(self, data):
        return self.__model__(**data)

class UserSchema(BaseSchema):
    __envelope__ = {
        'single': 'user',
        'many': 'users',
    }
    __model__ = User
    name = fields.Str()
    email = fields.Email()

user_schema = UserSchema()

user = User('Mick', email='mick@stones.org')
user_data = user_schema.dump(user).data
# {'user': {'email': 'mick@stones.org', 'name': 'Mick'}}

users = [User('Keith', email='keith@stones.org'),
        User('Charlie', email='charlie@stones.org')]
users_data = user_schema.dump(users, many=True).data
# {'users': [{'email': 'keith@stones.org', 'name': 'Keith'},
#            {'email': 'charlie@stones.org', 'name': 'Charlie'}]}

user_objs = user_schema.load(users_data, many=True).data
# [<User(name='Keith Richards')>, <User(name='Charlie Watts')>]

Raising Errors in Pre-/Post-processor Methods

Pre- and post-processing methods may raise a ValidationError. By default, errors will be stored on the "_schema" key in the errors dictionary.

from marshmallow import Schema, fields, ValidationError, pre_load

class BandSchema(Schema):
    name = fields.Str()

    @pre_load
    def unwrap_envelope(self, data):
        if 'data' not in data:
            raise ValidationError('Input data must have a "data" key.')
        return data['data']

sch = BandSchema()
sch.load({'name': 'The Band'}).errors
# {'_schema': ['Input data must have a "data" key.']}

If you want to store and error on a different key, pass the key name as the second argument to ValidationError.

from marshmallow import Schema, fields, ValidationError, pre_load

class BandSchema(Schema):
    name = fields.Str()

    @pre_load
    def unwrap_envelope(self, data):
        if 'data' not in data:
            raise ValidationError('Input data must have a "data" key.', '_preprocessing')
        return data['data']

sch = BandSchema()
sch.load({'name': 'The Band'}).errors
# {'_preprocessing': ['Input data must have a "data" key.']}

Pre-/Post-processor Invocation Order

In summary, the processing pipeline for deserialization is as follows:

  1. @pre_load(pass_many=True) methods
  2. @pre_load(pass_many=False) methods
  3. load(in_data, many) (validation and deserialization)
  4. @post_load(pass_many=True) methods
  5. @post_load(pass_many=False) methods

The pipeline for serialization is similar, except that the “pass_many” processors are invoked after the “non-raw” processors.

  1. @pre_dump(pass_many=False) methods
  2. @pre_dump(pass_many=True) methods
  3. dump(obj, many) (serialization)
  4. @post_dump(pass_many=False) methods
  5. @post_dump(pass_many=True) methods

Warning

You may register multiple processor methods on a Schema. Keep in mind, however, that the invocation order of decorated methods of the same type is not guaranteed. If you need to guarantee order of processing steps, you should put them in the same method.

from marshmallow import Schema, fields, pre_load

# YES
class MySchema(Schema):
    field_a = fields.Field()

    @pre_load
    def preprocess(self, data):
        step1_data = self.step1(data)
        step2_data = self.step2(step1_data)
        return step2_data

    def step1(self, data):
        # ...

    # Depends on step1
    def step2(self, data):
        # ...

# NO
class MySchema(Schema):
    field_a = fields.Field()

    @pre_load
    def step1(self, data):
        # ...

    # Depends on step1
    @pre_load
    def step2(self, data):
        # ...

Handling Errors

By default, Schema.dump() and Schema.load() will return validation errors as a dictionary (unless strict mode is enabled).

You can specify a custom error-handling function for a Schema by overriding the handle_error method. The method receives the ValidationError and the original object (or input data if deserializing) to be (de)serialized.

import logging
from marshmallow import Schema, fields

class AppError(Exception):
    pass

class UserSchema(Schema):
    email = fields.Email()

    def handle_error(self, exc, data):
        """Log and raise our custom exception when (de)serialization fails."""
        logging.error(exc.messages)
        raise AppError('An error occurred with input: {0}'.format(data))

schema = UserSchema()
schema.load({'email': 'invalid-email'})  # raises AppError

Schema-level Validation

You can register schema-level validation functions for a Schema using the marshmallow.validates_schema decorator. Schema-level validation errors will be stored on the _schema key of the errors dictonary.

from marshmallow import Schema, fields, validates_schema, ValidationError

class NumberSchema(Schema):
    field_a = fields.Integer()
    field_b = fields.Integer()

    @validates_schema
    def validate_numbers(self, data):
        if data['field_b'] >= data['field_a']:
            raise ValidationError('field_a must be greater than field_b')

schema = NumberSchema()
result, errors = schema.load({'field_a': 2, 'field_b': 1})
errors['_schema'] # => ["field_a must be greater than field_b"]

Validating Original Input Data

Normally, unspecified field names are ignored by the validator. If you would like access to the original, raw input (e.g. to fail validation if an unknown field name is sent), add pass_original=True to your call to validates_schema.

from marshmallow import Schema, fields, validates_schema, ValidationError

class MySchema(Schema):
    foo = fields.Int()
    bar = fields.Int()

    @validates_schema(pass_original=True)
    def check_unknown_fields(self, data, original_data):
        unknown = set(original_data) - set(self.fields)
        if unknown:
            raise ValidationError('Unknown field', unknown)

schema = MySchema()
errors = schema.load({'foo': 1, 'bar': 2, 'baz': 3, 'bu': 4}).errors
# {'baz': 'Unknown field', 'bu': 'Unknown field'}

Storing Errors on Specific Fields

If you want to store schema-level validation errors on a specific field, you can pass a field name (or multiple field names) to the ValidationError.

class NumberSchema(Schema):
    field_a = fields.Integer()
    field_b = fields.Integer()

    @validates_schema
    def validate_numbers(self, data):
        if data['field_b'] >= data['field_a']:
            raise ValidationError(
                'field_a must be greater than field_b',
                'field_a'
            )

schema = NumberSchema()
result, errors = schema.load({'field_a': 2, 'field_b': 1})
errors['field_a'] # => ["field_a must be greater than field_b"]

Overriding how attributes are accessed

By default, marshmallow uses the utils.get_value function to pull attributes from various types of objects for serialization. This will work for most use cases.

However, if you want to specify how values are accessed from an object, you can override the get_attribute method.

class UserDictSchema(Schema):
    name = fields.Str()
    email = fields.Email()

    # If we know we're only serializing dictionaries, we can
    # use dict.get for all input objects
    def get_attribute(self, obj, key, default):
        return obj.get(key, default)

Custom “class Meta” Options

class Meta options are a way to configure and modify a Schema's behavior. See the API docs for a listing of available options.

You can add custom class Meta options by subclassing SchemaOpts.

Example: Enveloping, Revisited

Let’s build upon the example above for adding an envelope to serialized output. This time, we will allow the envelope key to be customizable with class Meta options.

# Example outputs
{
    'user': {
        'name': 'Keith',
        'email': 'keith@stones.com'
    }
}
# List output
{
    'users': [{'name': 'Keith'}, {'name': 'Mick'}]
}

First, we’ll add our namespace configuration to a custom options class.

from marshmallow import Schema, SchemaOpts

class NamespaceOpts(SchemaOpts):
    """Same as the default class Meta options, but adds "name" and
    "plural_name" options for enveloping.
    """
    def __init__(self, meta, **kwargs):
        SchemaOpts.__init__(self, meta, **kwargs)
        self.name = getattr(meta, 'name', None)
        self.plural_name = getattr(meta, 'plural_name', self.name)

Then we create a custom Schema that uses our options class.

class NamespacedSchema(Schema):
    OPTIONS_CLASS = NamespaceOpts

    @pre_load(pass_many=True)
    def unwrap_envelope(self, data, many):
        key = self.opts.plural_name if many else self.opts.name
        return data[key]

    @post_dump(pass_many=True)
    def wrap_with_envelope(self, data, many):
        key = self.opts.plural_name if many else self.opts.name
        return {key: data}

Our application schemas can now inherit from our custom schema class.

class UserSchema(NamespacedSchema):
    name = fields.String()
    email = fields.Email()

    class Meta:
        name = 'user'
        plural_name = 'users'

ser = UserSchema()
user = User('Keith', email='keith@stones.com')
result = ser.dump(user)
result.data  # {"user": {"name": "Keith", "email": "keith@stones.com"}}

Using Context

The context attribute of a Schema is a general-purpose store for extra information that may be needed for (de)serialization. It may be used in both Schema and Field methods.

schema = UserSchema()
# Make current HTTP request available to
# custom fields, schema methods, schema validators, etc.
schema.context['request'] = request
schema.dump(user)