DbDataStruct.py - Definitions of the data structure used to update the database

This file contains a definition and supporting classes for a data structure used to export data from Sphinx to the Flask database for the books.


These are listed in the order prescribed by PEP 8.

Standard library

from collections import defaultdict, OrderedDict
from enum import Enum
from pprint import pformat
from pathlib import Path

Shared between Sphinx and Flask

There is a bi-directional exchange of data: Sphinx provides changes to the books/pages/questions/feedback, while the database then saves these changes and provides unique IDs for pages, which should be back-annotated to Sphinx. There’s also a lot of data at stake: deletions of books (!!!), pages, or even questions imply the deletion of all user data (answers, annotations) connected to those items, which should be avoided if at all possible.

Sphinx should store both the ID (if available) and the line number of the question / page directive, to make it easy for this program to do back annotation.

Pondering a data structure:

env.interactive_questions = InteractiveQuestions
    .gradebooks = {
        str(docname): [
            [int(level) str(docname)],
            [ ...repeats for all levels and docnames in the gradebook... ]
    .pages = {
        str(docname): Page
            .id_: int
            .line: int
            .index: int (only exists for pages that contain questions)
            .questions: Ordered{
                str(questionId): Question
                    .pointsPossible: int
                    .grader: GraderType
                    .feedback: {
                        str(answer): Feedback
                            .points: int
                            .string: str
    .source_paths = {  <-- Only in pickle file.
        str(docname): str(path_to_docname)
    .pageIds = {   <-- Only in Sphinx.
        str(id_): str(docname),
    .javaScript = { <-- Only in Sphinx.
        str(docname): str(javaScript)
    .feedback_raw = { <-- Only in Sphinx.
        str(docname): [
                .line: int
class BaseAutoassign:
    def __init__(self, **kwargs):

Proivde a nice pretty-print compatible representation.

    def __repr__(self):
        return ''.join('.{} = {}\n'.format(x, pformat(getattr(self, x))) for x in dir(self) if not x.startswith('__'))

class Feedback(BaseAutoassign): pass
class FeedbackRaw(BaseAutoassign): pass
class Question(BaseAutoassign): pass
class Page(BaseAutoassign): pass
class InteractiveQuestions(BaseAutoassign): pass

def pageMaker():
    return Page(id_=None, line=None, questions=OrderedDict())

While the approach at http://stackoverflow.com/a/18348004 seems great (for example, InteractiveQuestions.__new__.__defaults__({}, defaultdict(Page))), it doesn’t work: basically, this means that all created InteractiveQuestions share the same dict. Instead, use a helper function to create a new instace of a dict when constructing a new InteractiveQuestions instance.

def interactiveQuestionsMaker():
    return InteractiveQuestions(gradebooks={}, pages=defaultdict(pageMaker),
                                pageIds={}, javaScript=defaultdict(str),

Grader types

class GraderType(Enum):

Use an exact string comparison.

    exact = 0,

Trim leading and trailing whitespace, and replace all internal whitespace with a single space character. Ignore case.

    ws_ignore = 1,

The provided answer is a regular expression.

    regexp = 2,

The provided answer is a regular expression, with case-insensitive matching.

    regexp_nocase = 3,

The provided answer is a number, with a range of correct values.

    number = 4,

This is code. Compile and run it.

    code = 5,

Given a file, return the inline comment token based on the file extension.

def commentForExt(file_name):

    return {
        '.py': '# ',
        '.c': '// ',
        '.h': '// ',
        '.js': '// ',
        '.s': '; ',
        '.rst': '',
        '.css': '/* ',  # Will need some hand editing...
        '.ini': '; ',

Relative to the HTML output directory, the path to student source (which has all feedback and code solutions removed).


Add the appropriate line comment delimiters to the CODE_HERE_STRING.

def code_here_comment(file_name):
    return (_add_line_comment_delimiter(CODE_HERE_STRING, file_name)

Don’t add a comment character for the trailing newline.

        + '\n')

A string which replaces code solutions in student source code.

Add your code after this line.

Add your code before this line.

Given a string and the file name it will be added to, add the line comment delimiter followed by a space to each line of the string.

def _add_line_comment_delimiter(

The string to which line comment delimiters should be added.


The file name into which this string will be inserted.


    delim = commentForExt(file_name)

Add a comment for to every newline.

    str_ = str_.replace('\n', '\n' + delim)

Begin the string with a comment as well.

    return delim + str_

Shared between waf and Flask

Return the string needed to run a simulation.

def get_sim_str(

A string giving the MCU to simulate.


The ELF file to load and simulate.


The name of an output file for UART output.


    return (

In SIM30, type ? to get help. See also the manual.

Select the dsPIC33E. From the help: LD <devicename> -Load Device: dspic30super dspic33epsuper pic24epsuper pic24fpsuper pic24super

      'LD {}\n'

Load in the pic24_intro.elf. From the help: LC <filename> -Load COFF/ELF File

      'LC {}\n'

Have the simulator save UART1 IO to a file. From the help: IO [stdin [stdout]] -Input/Output On (use nul if no stdin and/or stdout)

      'IO nul {}\n'

Reset the processor. From the help: RP -Reset processor POR


Set a breakpoint at the end of the program (the label done). From the help: BS <location> ...[locations] -Breakpoint Set

      'BS done\n'

Run the program. From the help: ; E  -Execute


Quit. From the help: Q  -Quit

      'Q\n').format(sim_mcu, elf_file, uart_out_file)