Complex Data Model Generation with mobo
Simon Heimler (www.fannon.de )
University of Applied Sciences Augsburg, Germany Computer Bauer GmbH, Munich, Germany
Hello! My name is Simon Heimler and I'm from the University of Applied Sciences in Augsburg, Germany.
In this session I would like to introduce you to a project called mobo.
As you might guess from the program title its project that helps with complex data model generation.
Preface
Prototype in action
Uses a very different approach than current solutions.
Open Source and open to discussion!
Appetizer first
But before we're heading into the technical details, i'll give you a little appetizer first:
Your browser does not support the video tag.
Lets begin with a typical Semantic Form. I realize that I've implemented an attribute wrong.
So how come mobo into play? Lets head over to my text editor and search for a field "loginName"
As you see its a simple data file. Lets change the form field. While we're at it we could also provide an default value for this field.
The moment I hit save mobo will realize the change in the background. [STOP]
As you see mobo recalculates the data model and realizes that 25 wiki sites have to be changed and uploaded. It does that in a few secons total.
Lets head back to our browser. Now the field is updated.
Outline
Using JSON Schema...
...for an object oriented...
...model development...
...workflow...
...in Semantic MediaWiki.
Conclusion & Live-Demo
I have thought about a more detailed title for this presentation and come up with one that even serves as the outline for the presentation:
USING JSON Schema
Lets start with the base technology behind this project: JSON Schema.
Who of you is familiar with JSON Schema?
Only a few? Lets change that!
JSON Schema
Data format for describing data formats / objects
Easy to read/write for both humands and machines
Published draft at IETF (Internet Engineering Task Force)
Similar to XML Schema, but based on JSON
What is JSON Schema about?
Example File
{
"firstName": "Simon",
"lastName": "Heimler",
"age": 27
}
Lets begin with a very simple data example.
This is a JSON File or JavaScript Object describing some facts about me.
Example JSON Schema
JSON Schema is using the same data format JSON to meta describe the object.
You see that we are talking about a person here, that has the properties firstName, lastName and age. But there's more: FirstName and lastName are required and age should be at minimum 0.
This is a very simple example. Of course it's possible to describe much more complex data structures.
Use cases
So what is JSON Schema good for? There are a few Use Cases:
The most obvious one is to use it for validation...
As far as I know mobo is the first project to use JSON Schema for data model generation.
json-schema.org
http://json-schema.org/
JSON Schema has its own website with documentation and a list of libraries that make use of it. So if you're interested in this, have a look.
object-oriented
mobo makes use of object orientation.
Object orientation
More complex SMW models tend to have much redundancy
This can lead to inconsistency - fast
Inheritance can keep the model DRY (Dont Repeat Yourself)
The OO paradigma is simple and well understood
Let me give you a few reasons why using object orientation could be a good idea:
Redundany - at least this was my experience!
Object-oriented JSON Schema
JSON Schema uses "$ref"
to include and reuse schemas
BUT: There is no inheritance behaviour defined!
Sadly, JSON Schema does not support inheritance in a standardized manner.
Own implementation
Implemented custom, more explicit "$extend"
attribute which does proper inheritance
Through "$extend"
the relationship between the objects are defined, too
This leads to a (graph) database model that fits SMW well
But thats not a big problem. JSON Schema is built to be easily extensible. So I implemented my own, custom $extend attribute...
SMW Specifics
Uses Fields, Models and Forms that roughly equivalent SMWs Attributes, Templates and Forms
Additional schema attributes that reflect SMW/SF settings
Form settings can be defined as low as in Fields and are inherited / overwritten on the way up
Of course there are some specific needs of SMW.
Concept
Object-oriented modeling concept
To make this more visual, lets take a look at this diagram.
Lets begin on the right hand side: There's a Circle Form that implements a Circle Model and a Quality Model.
The Circle Model inherits from the abstract _Shape Model, which has the attributes x and y. The Circle also implements a new attribute radius.
Everything is inherited and overwritten from left to right.
/model/_Shape.json
This is the abstract Shape model. It has both x and y coordinates which are defined as integers. b
/model/Circle.json
{
"$extend": "/model/_Shape.json",
"title": "Circle",
"type": "object",
"properties": {
"radius": { "$extend": "/field/radius.json" },
},
"required": ["x", "y", "radius"],
"abstract": false
}
Now lets take a look at the Circle Model. It extends from _Shape and inheritc all properties. Some are overwritten, like the title for example. It inherits the properties from Shape and adds the radius. The radius attribute is inherited too.
/field/radius.json
{
"title": "radius",
"description": "The radius of a shape",
"type": "number",
"minimum": 0,
"smw_form": {
"input type": "text with autocomplete"
}
}
This is the radius field. As you can see the Semantic Forms Display Type is defined as low as in the attribute itself.
model development
So how does model development look like with mobo? There are a few tools involved.
The model is a collection of folders and JSON files on your filesystem.
Use your favorite text editor and VCS!
It is possible to split the model into several modular, reusable sub-projects.
With VCS its possible to go back and forward in the development state in seconds.
With VCS the model can be developed and synced between many persons.
Web based GUI for browsing/viewing the final, inherited model state
... and the wikitext results
Interactive model graph explorer
Sometimes its useful to have a visual representation of your model so you can spot problems from bird's-eye view. This is an interactive model explorer that lets you click through your model.
This graph is also auto generated, but requires some custom layouting through Gephi, a Graph Analysis Tool.
workflow
How does the typical workflow of using mobo look like?
GETTING STARTED
npm install mobo -g
Installs mobo as a global NPM (Node.js) package
mkdir new_project && cd new_project
Creates a new directory and enters it
mobo --init
Initializes a new (barebone) project
nano settings.json
Adjust settings.json (login data!)
First: Lets get started !
Using mobo
Enter mobo in your project directory and its executed in the interactive mode. The Webapp is served at the localhost.
Interactive Workflow
A file of the development model is changed / added
mobo detects the change and re-generates the model
Model is validated. Errors and warnings will be displayed
The result is compared to last upload state and only the changed sites will be uploaded via a Bot account
This is how the output of mobo can look like.
The yellow lines are warnings about inconsistency in the model.
After that mobo lists all sites that are changed through the last update.
At last all uploads are listed.
As you can see, this process - including upload - completes in seconds.
Semantic MediaWiki
So how does this all relate to Semantic MediaWiki?
Target Installation
Semantic MediaWiki and Semantic Forms are required
HeaderTabs Extension is supported
Visual Editor TemplateData Extension is supported; will be used for template documentation
Form with HeaderTabs
This is an example form that was completely generated by mobo.
The form uses HeaderTabs.
Site with HeaderTabs
This is the site resulting from the form entry. In this tab a Router was added.
Documentation and usage snippet is auto generated
mobo also generates documentation and snippets how templates or queries are to be used.
A report is generated and uploaded
After each upload mobo creates and uploads a report to the wiki.
Very useful toolset for my own model development process
Great at keeping the model consistent, possible to update dozends of wiki sites in a minute.
Works well with prototyping / iterative development process.
SMW/SF doesn't support some features that would come cheap (HTML5 front-end validation!)
Completely independent from SMW/SF
Adds to the learning curve of SMW
Model needs to have a certain complexity to make using the toolset viable
No GUI for editing
Project state
Working Prototype
Currently tailored to the specific needs of a specific project,
but this might change.
Open Source at github.com/Fannon/mobo
DEMO
Now I'll give you a little live demo. Feel free to ask questions now!
Your browser does not support the video tag.
Questions?
We still got some time left. So if there are any more questions, feel free to ask them now!
Thanks for listening!
If you've got feedback or questions, just talk to me!
www.fannon.de/p/jsonmodel
Thanks for listening to my session!
If you've got some more feedback or questions, just talk to me later.
OWL is more flexible and powerful
OWL supports reasoning / inference
OWL is much more difficult than JSON Schema
Simple things like hard validation are difficult with OWL. It requires another complex layer on top of a already complex system. (e.g. Pellet Integrity Constraints )
My personal philosophy
The foundations should be as easy and barebone as possible while beeing still highly flexible / modular
More complex features should come on top of that
Excursus
How about Page Schemas?
Schema editor in the browser
Easy to understand / use
Requires no additonal technologies
Tightly integrated in SMW
Supports no inheritance
No Validation
Slower Workflow