Batch Data Entry: the first step towards data literacy

All triggers are bulk triggers by default, and can process multiple records at a time. You should always plan on processing more than one record at a time. –Salesforce

Database applications are typically split into data entry and reporting. For data entry they offer different ways of finding the record you wish to create or change. For reporting they offer different ways of filtering and summarizing your records.

Typically analytics is applied to reporting, but users spend much more time entering and updating data than they do looking at reports. The idea that all users should be able to do analysis of the data means that we have to convince users who do the bulk of their work in data entry screens to also go look at reports or dashboards. This is probably going to be viewed as “extra” work on their part and it will be hard to convince them.

We can borrow a metaphor from Salesforce and make all data entry deal with “batches” instead of “records”. We place on the same form a way to select a batch of records. Then a way to select one of the records to view or edit its details. For every “batch” of records we display summary statistics on the same form. The user must select a batch in order to select a record, and every time they select a batch they also see the statistics associated with that batch.

Email offers an everyday example of dealing with things in “batches”. Your inbox is a batch of emails. If you search for an email address you see a batch of emails that match your search. When you click on a record in a batch you then see the full record. Email clients sometimes¬†use a different screen to display the full record… but our model is going to try to display both the batch and the selected record on the same form.

One of the big problems with trying to improve your organization’s data culture is that thinking about data¬†in terms of its context is much harder than thinking about it in terms of the content of a particular record. A “batch” based interface lets the user’s come to grips with their data’s context by constantly exposing them to it as they do their daily work. You could compare it to the list of ingredients on the side of cereal boxes as opposed to the advertising copy on the front of the box. Using a “batch” interface gradually improves the user’s abilities to “think” about their data. You could think about it like a sports team that spends time before every practice doing exercises. The exercises strengthen the team members and make them more able to learn the skills demanded for their sport.

One immediate consequence (in the applications I have built in this way) is that when the users are filtering a batch of data they often immediately see some records which should not be in that batch. For example, they are filtering for active members, but they see a name in the batch they know is not an active member. Because they can click on the offending record and “fix” it, seeing their data in terms of “batches” means that the batches will become “better” as the records in them become more correct.

A “batch” style interface can be constructed in any programming environment. I have posted some wire-frame drawings that may help in this post.



Leave a Reply

Your email address will not be published. Required fields are marked *