David Smith's Blog

Customising your Python REPL

This blog post will show you how to customise your Python REPL by running a Python file before the interactive prompt is loaded. This is achieved using the `PYTHONSTARTUP` environment variable.

This blog post will show you how to customise your Python REPL by running a Python file before the interactive prompt is loaded. This is achieved using the PYTHONSTARTUP environment variable.

Initial set up.

By way of an initial example, you may wish to always have the pprint function available when the REPL loads.

To achieve this we must first have a python file that will be run before the REPL loads.

Create a file:

touch .pythonstartup.py

Populate file with code to import pprint:

# import pprint
from pprint import pprint

And finally, set the PYTHONSTARTUP environment variable to point at our Python file:

export PYTHONSTARTUP=~/.pythonstartup.py

To test if this is working, launch a Python REPL and check the function is available.

>>> pprint
<function pprint at 0x7f4c0e9ebb00>

Alternative shell

Alternative shells are available such as IPython or Ptpython. While these can be launched via their own commands you could also use your PYTHONSTARTUP script to try to launch them as your default shell.

We'll use Ptpython as an example. The .pythonstartup.py file becomes:

from pprint import pprint

import sys
try:
    from ptpython.repl import embed
except ImportError:
    print("ptpython is not available: falling back to standard prompt")
else:
    sys.exit(embed(globals(), locals()))

By passing globals() and locals() to embed() this allows the imported pprint() function to be available in the new shell.

If you were using IPython this could be achieved by using start_ipython() and using the user_ns parameter to initalise the IPython namespace.

Django shell

The django-admin CLI has a shell command.

This has an --interface option which can be used to launch the shell with different shells (ipython, bpython, python).

Using the script above the PYTHONSTARTUP the Ptpython shell will be launched by default. If you wished to instead use the default Python shell this could be achieved by specifying the Python shell and using the --no-startup option:

django-admin shell --interface python --no-startup

Conclusion

Using a custom PYTHONSTARTUP script can be used to import frequently used functions so they are always available and allow use of your favourite Python shell with Django even if it doesn't have a built-in option.

Generational Sustainability

In community-led organizations, I believe that for long-term sustainability, we need to do more to encourage younger folks to become involved and take on leadership positions. Younger individuals won’t break a community, but not encouraging them may well do so.

A story:

As a young person, I was a member of a youth group. I thoroughly enjoyed my time there, meeting new friends, learning new skills and participating in exciting activities. Looking back now, I realize that I learned a lot through these experiences.

As the years passed, I started volunteering as a young leader, giving back to the community by working alongside the adults who were leaders when I was a youth member. During this time, I completed the formal training for the organisation. Having been involved with the organization for about 15 years, most of it came naturally. However, it provided me with the opportunity to learn new things. I encountered ideas and concepts that were new to me, and I realised that young people can bring fresh perspectives and enthusiasm.

Around this time, the organization recognized they had a problem. Many of the leaders were aging; many had been in their roles for 20+ years, some 40+. The organisation needs new young leaders to ensure long-term sustainability, they said. Everyone nodded in agreement.

I continued volunteering in the same role for the next decade or so. Life went on — education, finding employment, and so on. The enthusiasm for volunteering became more challenging as I had less time. Those in senior positions had accumulated many years of experience. While they were always grateful for the help enacting change within the organisation is challenging (committee meetings were particularly tiresome). Around the same time, I had my first child and used this as my excuse to step back from volunteering.

A decade passed.

My children are now members of youth organisations. The folk in senior positions are the same people that were leaders when I was a youth member (Yes, I'm old). This is not an example of a single group within this organisation or even specific to a single organisation. I see it across groups and organisations in my local area.

It's evident that transition needs to happen to a younger generation. Preferably sooner rather than later. There's already consequences being seen of this not happening.

The pandemic resulted in groups closing, some did not reopen. A youth movement needs to engage people at a young age, so they keep on being members for years to come. Attracting members at an older age is challenging. Therefore, due to this one section closing the sections aimed at older ages have also closed as they saw member numbers fall away.

I wonder if younger folk had been encouraged and given the opportunity to take on senior leadership positions this would not have happened. Those with experience can still be there to support and coach, but not everything needs to be on their shoulders.

For long term sustainability we need to encourage and support the next generation to take on senior positions. Older folk, I'm not saying that younger folk don't want you about any more, they value and need your experience and guidance. But everything you have built and contributed to may fade away if the next generation doesn't, at some point, step up and take the lead.

How to find a ticket on Django's trac

This is my guide on how I go about finding a ticket on Django's trac to work on.

Django's issue tracker

Django's issue tracker is hosted at code.djangoproject.com. This contains the entire history of ticket's opened for Django. At time of writing this is some 18 years ago! This is a valuable resource when working on Django as you are able to research previous decisions.

From the summary page clicking "View Tickets" will show a listing of Django's tickets which are not closed. As of November 2023 Django currently has 996 open tickets. This is alot! Luckily, Trac (the software which Django's issue tracker uses) has a number of filtering options to help this down.

Filter to accepted tickets

Django's triaging process means that not all open tickets are currently accepted. Some may be marked to be reviewed in the future, for example a ticket may be proposed to use a feature in a new release of Python. However, this will only be possible to achieve in the future once that feature is in all versions of Python which Django currently supports.

To do this click on the filter category box and select triage stage:

This option will then appear in the filter box. Then select "Accepted" from the choice of options. You'll need to click the "Update" button to apply this filter selection.

Filter by component

This'll bring the number of tickets down to ~930 tickets. Better, but we're not there yet.

The next filter I reach for is component. This is used to categorise tickets into which area of Django they relate to. Here's a chart which shows the number of tickets by component.

This chart shows around 40% of tickets are for Django's ORM or Admin. Choosing any other component than these means the tickets will be reduced to a more manageable number. At this point I suggest reviewing the list of components and trying to find one that you're interested in.

Ticket Investigation

Say you chose the "Template System" as the component. This reduces the number of tickets down to just 24. This is a more manageable number to have a more detailed review of. At this point I would suggest starting to look through tickets. In particular:

Ignore tickets which are actively being worked on. I'd judge actively as meaning anything in the last ~3 months.
Review the tickets for descriptions of things I have any knowledge on.
Bear in mind that tickets with higher numbers are more recent. It's likely that ticket #34981 (Opened Nov 2023), is likely easier to solve than #373 (Opened 18 Years ago).

Standing on the shoulders of giants

At this point you'll start to get a feeling of the types of tickets there are to be tackled. Clicking on the description will take you to the ticket page giving a summary of the issue and the previous discussion that's taken place.

My tip here is to look for tickets that have entries against "Pull Requests". There are many tickets which have had previous Pull Requests opened and reviewed. Some of these may just need updating for review comments, additional documentation or adding additional test coverage.

Finding something where previous work has been conducted can ease your contribution as you can build upon previous work. The task may be less daunting than a ticket where it may be slightly unclear what the required bug or fix is.

Assign a ticket to yourself

At this point you are very knowledgeable about a certain ticket or category of tickets. Once you have selected a ticket, do assign it to yourself. To do this you'll need to register / log in and complete the "Action" form at the bottom of the page.

Next Steps

I'd start with referring to Django's contribution guide in the documentation. In addition Django's forum and Discord (see invite link on the community page) are channels where there are folk who will be willing to help you.

How to ask for help

Something that took me a long time to learn is that other folk won't know the answer, or even be familiar with an issue. You will be the expert. What is helpful is if you can summarise the issue and your thoughts on what is required. Specific questions are more likely to get a response than general ones. General questions about tickets are tricky to answer as it's highly likely no one yet knows the answer, otherwise it would have been fixed all ready!

Good luck in your contribution to Django 💚

Building a Bootstrap styled form in vanilla Django.

A walk through of building a bootstrap 5 styled form using only Vanilla Django. Exploring per-project, per-form and per-field customisations.

I tooted a little while back about how using django-crispy-forms and it's Layout class could ease writing forms. It showed how a minimal Layout could result in a nice looking form with some complex layouts such as two input boxes on the same row.

Here's the content of that toot.

django-crispy-forms layout:

Resulting form:

I followed this up with a bit of a personal challenge. What would it take to recreate this form using vanilla Django. No 3rd party packages allowed.

This post follows my thought process as I worked through this challenge and highlights the challenges I faced. I'd love to hear feedback on these, what am I doing wrong and if there anything here that Django itself could improve.

Let's go!

Create the base form

The first step was to create a test project, I used a single project folder layout putting everything in the urls.py file. Here's the relevant section that shows the base form. I'm also using Django 5.0 (a few weeks away from release at time of writing) as it includes a number of new features that I'll be making use of.

from django import forms


# 🎉 Django 5.0 allows options to be defined as a mapping. 🎉
days = {
    1: "Monday",
    2: "Tuesday",
    3: "Wednesday"
}


class BootstrapForm(forms.Form):
    name = forms.CharField(max_length=50)
    email = forms.CharField(max_length=50)
    appointment_date = forms.ChoiceField(
        choices=days,
        widget=forms.RadioSelect(),
        help_text="Select which day you would like your appointment. We're only open Monday-Wednesday."
    )
    attachment = forms.FileField(
        required=False,
        help_text="Please provide any files you wish us to review prior to your appointment.",
    )

I added a view that returns a TemplateResponse. The template used contains the Bootstrap CSS and JavaScript and renders the form using {{ form }} along with some input buttons. Rendering the whole form using {{ form }} is a key aim here. Here's the part of the template showing the form.

<form action="/" method="post">
    {% csrf_token %}
    {{ form }}
    <input type="button" value="Submit" class="btn btn-primary">
    <input type="button" value="Cancel" class="btn btn-danger">
</form>

By default this will use Django's as_div template, rendering each field in a series of <div> elements. We can now begin to customise the form.

Add css classes to a field's `<input>`.

Let's begin styling our form by adding classes to the <input> elements.

Form widgets are Django's representation of <input>s. Classes can be added to these by using the Widget.attrs argument. Bootstrap has requires form-control for most widgets but form-check-inputs for radios and checkboxes. Here's an example of how the form can be updated to add these classes.

-    name = forms.CharField(max_length=50)
+    name = forms.CharField(max_length=50, widget=forms.TextInput(attrs={"class": "form-control"}))

Create a custom field template

A BoundField represents the whole field including its label, input, help text and errors. Django 5.0 added the ability to render a BoundField using its as_field_group () method. The template used can be set project-wide, per-field or per-instance.

I began by creating a copy of Django's field.html template in my project. To use this template project-wide create a custom FORM_RENDERER:

# urls.py
from django.forms.renderers import TemplatesSetting


class CustomFormRenderer(TemplatesSetting):
    field_template_name = "forms/field.html"

and in your settings.py file define the FORM_RENDERER setting.

# settings.py
FORM_RENDERER = "bootstrap_form.urls.CustomFormRenderer"

Styling `<label>` tags.

A custom field template eases adding additional styles and changing the HTML layout. Let's look at this by adding Bootstrap's .form-label to <label> and <legend> elements.

Django allows css classes to be added to labels if the field is required or has errors with Form.error_css_class and Form.required_css_class. However, the requirement here is to always add a class to a label irresepective of form state. An alternative approach is required.

The default template renders the label with {{ field.label_tag }}, and although the label_tag function allows attrs to be added you can't pass arguments to the function with the Django Template Language. We could override the default label.html template, but as it is a small change I included the customisation in the field.html template. Here's the customisation I made to the <label> tag, a similar change was made for <legend>.

- {{ field.label_tag }}
+ <label class="form-label">{{ field.label }}</label>

Styling help text.

Changing the style of help text is straight forward. The field template is updated to change the default .helptext to bootstrap's .form-text.

- {% if field.help_text %}<div class="helptext"{% if field.id_for_label %} id="{{ field.id_for_label}}_helptext"{% endif %}>{{ field.help_text|safe }}</div>{% endif %}
+ {% if field.help_text %}<div class="form-text"{% if field.id_for_label %} id="{{ field.id_for_label}}_helptext"{% endif %}>{{ field.help_text|safe }}</div>{% endif %}

Using a per-field template.

Django's default template for radio inputs is to render it as an unordered list using <ul> and <li> elements. Bootstrap 5 requires the input to be rendered alongside the <div>, in addition the <div> wrapping the <label> and <input> also needs .form-check. See docs

Changes are needed both to the widget and field templates. I began by creating a per-field template and defined this in the form:

     appointment_date = forms.ChoiceField(
         choices=days,
         widget=forms.RadioSelect(),
+        template_name="forms/radio_field.html",
         help_text="Select which day you would like your appointment. We're only open Monday-Wednesday."
    )

I used the field template to customise the layout of both the field and the widget by using the more granular control available in Django for widgets with choices. This allows wrapping of each <label> and <input> pair in a <div> with the required css classes. Here's the full template:

<fieldset>
  <legend class="fs-6">{{ field.label }}</legend>
{% if field.help_text %}<div class="form-text"{% if field.id_for_label %} id="{{ field.id_for_label}}_helptext"{% endif %}>{{ field.help_text|safe }}</div>{% endif %}
{{ field.errors }}
{% for choice in field %}
    <div class="form-check">
    <label for="{{ choice.id_for_label }}" class="form-check-label">{{ choice.choice_label }}</label>
    {{ choice.tag }}
    </div>
{% endfor %}
</fieldset>

Customising the form template.

The fields are looking much better, but we need to customise the layout of the field groups to add some space between them and to have the name and email fields to be on the same row. I copied the div.html template to my project and added it to my form renderer. The template can also be set per-form by defining template_name on the form class.

 class CustomFormRenderer(TemplatesSetting):
     field_template_name = "forms/field.html"
+    form_template_name = "forms/form.html"

Adding classes to each field group.

The fields are currently too close together. To give them some space .mb-3 can be added to <div> wrapping the field group. The relevant line from the template is:

<div{% with classes=field.css_classes %}{% if classes %} class="{{ classes }}"{% endif %}{% endwith %}>

field.css_classes returns the classes for fields which are required of have errors mentioned above. In addition it takes an optional argument to add additional classes. Accessing this method from a Django template is not straight forward. However, given this is already a template .mb-3 can be added directly to the template.

<div{% with classes=field.css_classes %} class="mb-3{% if classes %} {{ classes }}{% endif %}"{% endwith %}>

Customising the form layout

The form we're working towards has the name and email fields on the same row. The custom per-form template can be edited to achieve this. The new as_field_group() method introduced in Django 5.0 eases this as the form can be laid out by accessing each field on the form rendering it as a field group. This avoids needing to write out each element for each field of your form.

The core part of the form template now becomes:

<div class="row">
    <div class="mb-3 col">
    {{ form.name.as_field_group }}
    </div>
    <div class="mb-3 col">
    {{ form.email.as_field_group }}
    </div>
</div>
<div class="mb-3">{{ form.appointment_date.as_field_group }}</div>
<div class="mb-3">{{ form.attachment.as_field_group }}</div>

Customise form errors

The form is now looking pretty similar to our target (radio's are not on the same row -- you'll have to take inspiration from the crispy-form template pack for that 😊). However, there's no errors on this form. If the submitted form returned validation errors these would be styled using Django's defaults. We'll need to customise these to work with Bootstrap.

Django uses a class to hold information about form errors themselves and how to render them. A custom class can be used to change the template used to render errors.

To create a custom error class:

from django.forms.utils import ErrorList

class CustomErrorList(ErrorList):
    template_name = "forms/errors.html"

The template renders each error in a span with the .invalid-feedback class.

{% for error in errors %}
    <span class="invalid-feedback"><strong>{{ error }}</strong></span>
{% endfor %}

To use it in your form you need to define it when the field is created. I overrode the forms __init__ method to ensure that this custom error list is always used.

class BootstrapForm(forms.Form):
    def __init__(self, *args, **kwargs):
        kwargs.update({'error_class': CustomErrorList})
        super().__init__(*args, **kwargs)

While the form will render HTML for Bootstrap 5 style errors they currently don't appear!

For errors to be shown two things are required:

The field must have the .is-invalid class
The error must come after the field and is must be a sibling.

Adding error styles to widgets

Error styles were discussed above and can easily be added to surrounding <divs> or a <label>. For Bootstrap the error class needs to be added to the <input>.

Django allows additional classes to be added to a widget at render time when using the attrs argument of BoundField.as_widget()

We need to pass in additional error classes to this method when the field has errors. While it's not possible to do this directly from Django a custom template tag can be created to achieve this.

Here's my template filter. It takes in a field and some css classes. It adds the classes provided to any classes already on the field (such as .form-control) and renders the field with the combined classes. Note, this is a minimal example and is not well tested like some solutions in 3rd party packages.

@register.filter
def add_class(field, css_class):
    if widget_class := field.field.widget.attrs.get("class"):
        css_class += " " + widget_class
    return field.as_widget(attrs={"class": css_class})

In the field template we can use this to add the .is-invalid class when the field has errors.

{% if field.errors %}{{ field|add_class:"is-invalid" }}{% else %}{{ field }}{% endif %}

This works great for our name and email fields. However, it doesn't work for the RadioSelect widget!

The <input> for the radio is rendered using {{ choice.tag }} in the template. Adding the above filter here will result in an error. This is because choice.tag is already a string, with choice being a BoundWidget (designed for use with widgets like radios) not a BoundField.

We can update the template filter to also work for BoundWidget.

 @register.filter
 def add_class(field, css_class):
+    if isinstance(field, BoundWidget):
+        # copy data to avoid changing it, this method can be called multiple times
+        data = copy.copy(field.data)
+        if widget_class := data["attrs"].get("class"):
+            data["attrs"]["class"] = css_class + " " + widget_class
+        else:
+            data["attrs"]["class"] = css_class
+        context = {"widget": {**data, "wrap_label": False}}
+        return field.parent_widget._render(field.template_name, context, field.renderer)
     if widget_class := field.field.widget.attrs.get("class"):
         css_class += " " + widget_class
     return field.as_widget(attrs={"class": css_class})```

As per the warning above. This isn't well tested! The key takeaway is that some kind of template filter/tag is how I'd go about solving this. This can be used in our template like this.

{% if field.errors %}{{ choice|add_class:"is-invalid" }}{% else %}{{ choice.tag }}{% endif %}

Changing where errors are rendered in the form

The above section solved the need to add .is-invalid to <input> with errors. Step two is to address where the errors are rendered, they need to be directly after the <input>.

For our standard fields this is straight forward. {{ field.errors }} can be moved from above the input to below.

It's more complex for radios, the error has to follow the last <input> in the group. This is inside a for loop. The Django Template Language allows us to use forloop.last to check if it's the last item in the collection.

Labels and inputs also need switching round for bootstrap errors to display correctly. The core part of the radio field template becomes:

 {% for choice in field %}
     <div class="form-check">
     {% if field.errors %}{{ choice|add_class:"is-invalid" }}{% else %}{{ choice.tag }}{% endif %}
     <label for="{{ choice.id_for_label }}" class="form-check-label">{{ choice.choice_label }}</label>
    {% if forloop.last %}{{ field.errors }}{% endif %}
     </div>
 {% endfor %}

Make use of template filter for all widget classes.

The introduction of a template filter to add classes to widgets in templates means that the styles previously added in the form (which can get quite repetative) can be moved to the template.

For example, the need to define a widget and attrs dict for each field can be removed:

class BootstrapForm(forms.Form):
     ...
-    name = forms.CharField(max_length=50, widget=forms.TextInput(attrs={"class": "form-control"}))
+    name = forms.CharField(max_length=50)

... and can be added for all widgets in the template:

- {% if field.errors %}{{ field|add_class:"is-invalid" }}{% else %}{{ field }}{% endif %}
+ {% if field.errors %}{{ field|add_class:"form-control is-invalid" }}{% else %}{{ field|add_class:"form-control" }}{% endif %}

What about using a custom BoundField?

The key class in rendering of form fields is the BoundField. A custom BoundField could be created which could add custom css classes to widgets or labels. However, to use a custom BoundField every field's get_bound_field() method would need to be overriden. I think if this could be done at the form or project level what would ease this approach.

Conclusion

Reproducing the form using Django's core features proved to be mostly template work making use of per-site, per-form and per-field customisations. Some areas that were more challenging were:

Customising errors where a per-form error class needs to be provided.

Here I'd like to see the ability to set a project-wide template on the form renderer, if that's possible. I think most folk would want to customise the ErrorLists template, and not it's other features.

Customising classes for <inputs> which can be somewhat repetitive to add to a form and are more complex to add when they are dependant upon the form state, for example when the field has errors.

This seems harder, I'm not sure if there's a change that I'd like to see in Django itself for this. Maybe my solution to this problem isn't the correct one. Let me know your thoughts!

David Smith's Blog

Customising your Python REPL

Initial set up.

Alternative shell

Django shell

Conclusion

Generational Sustainability

How to find a ticket on Django's trac

Django's issue tracker

Filter to accepted tickets

Filter by component

Ticket Investigation

Standing on the shoulders of giants

Assign a ticket to yourself

Next Steps

Building a Bootstrap styled form in vanilla Django.

Create the base form

Add css classes to a field's <input>.

Create a custom field template

Styling <label> tags.

Styling help text.

Using a per-field template.

Customising the form template.

Adding classes to each field group.

Customising the form layout

Customise form errors

Adding error styles to widgets

Changing where errors are rendered in the form

Make use of template filter for all widget classes.

What about using a custom BoundField?

Conclusion

Add css classes to a field's `<input>`.

Styling `<label>` tags.