Implementing CRUD Wrappers — Reading Data

We are nearly at the end of our lengthy tour of CRUD wrappers! The last article looked at removing data, and today we will take a look at reading data. The first question you might be asking yourself, is, “why would I want a CRUD wrapper to read data?” (besides hitting the “R” in “CRUD” for completeness) and that is a truly good question.

I feel so off brand here, I tried my best to shoehorn my way into this picture and there wasn’t a good way to do it, and the alternative was to try buying punchcards off eBay and I really didn’t feel like waiting another week for those to show up before publishing this…

Let’s look at some pros and cons:

Pros

• You can log/audit who/when/where a read occurred. This is needed in applications under heavy regulatory scrutiny, such as HIPAA-compliant apps, ones that handle private information, financial data (PCI regulations come to mind here), etc.
• You can add security checks directly in the CRUD wrapper to guarantee that access rules are followed.
• If there are complex joins, filters, and other logic needed to read data (perhaps you have a record versioning system, or need to work with data from a third-party data source, or present a normalized data record from many tables at once), centralizing it in a CRUD wrapper makes sense.
• Can easily set caching.
• Caching on a single Action, or a specific query within that Action, is going to be much more efficient than, say, caching every place where you run a particular query because it will be less cached items in memory.
• Completeness: we are calling these “CRUD” wrappers, not “CUD” wrappers, right?

Cons

• Performance hit #1: the query optimizer can’t optimize the queries. This is especially brutal if any of the Entities have binary data or long fields. You can mitigate this by having situation-specific CRUD wrappers, and by putting large fields (binary data, big text fields) in separate entities. That said, changing the data model from a natural, organic one to an artificial structure for performance reasons… “feels bad”.
• Performance hit #2: you are passing records by value through the call stack. You can partially mitigate this by minimizing the data you pass by having situation-specific CRUD wrappers.
• Performance hit #3: you are sending all of the results to the caller, unless you implement a paging system, which is more effort.
• Performance hit #4: if you want to provide the row count, you need to either always do it (a performance hit in case it isn’t needed), or have a separate Action or toggle in the Action to provide it.
• Extra work to write the CRUD wrappers, and even more work if you decide to write situation-specific CRUD wrappers.
• You are bloating your code base with a massive amount of code, much of which may only be used in one place.

Needless to say, CRUD wrappers for reading data are have a lot of drawbacks and few advantages. You do see from time-to-time a specialized one of these, such as, say, a User_GetName() function that looks up a user and outputs a full name, or Tenant_GetIsActive(). These are forms of lightweight, specialized CRUD wrappers for reading, and are often preferred because they provide information that is provided multiple times per page request, so having the function (especially with caching) can be a big performance improvement.

But writing CRUD wrappers for reading, particularly data that you want to use in a table or a list is a lot of effort, typically needs to be custom-crafted to a particular scenario to ensure the best possible performance, and slows down your development time and clutters your code to make so many extra functions. You end up looking at your code and realize that you have sacrificed so many of the benefits and advantages to OutSystems, and you have escalated your development from “low code” to “medium code”.

I only write CRUD wrappers for reading to work with basic lists and tables if I absolutely must provide logging and auditing functionality to comply with a specific reason such as regulations or privacy.

There are simply too many concerns around them and not enough benefit to make them a standard practice. Along the same lines, because there are different reasons and different needs for writing one, I do not feel really comfortable giving guidance on them. I hope this does not feel like an excuse, but there is just no way to write a generic CRUD wrapper for reading that will make sense outside of a few specific needs.

In general, I will say that they will need some method of passing in search criteria, you will probably want to output a structure that has the exact fields you need to return (to make it clear to the caller that “this is the data you will be getting” rather than thinking that certain fields have values, but are missing them due to performance or security issues), and you should be certain that your logging, if you do it, does not have an impact on performance. The “LogRecord” Action in the “AsynchronousLogging” Extension will let you log to your custom log Entity, but the calls will be put into a high performance message queue so they do not block your code waiting for the chance to INSERT data to a common log table.

And again, there are places where a CRUD wrapper for a read makes a lot of sense. The “Fetch Data From Server” system in the Reactive Web Apps and Mobile Apps… well, that’s really just a screen-specific CRUD wrapper, isn’t it, when you think about it? While that lacks reusability, in so many cases it may be better to just add the needed logging or auditing or security filters there, and eliminate all of the performance concerns, then it is to try writing a true CRUD wrapper for reads.

Our next article on CRUD wrappers should also be our last one, and it explains how to take the CRUD wrappers that you have spent the time and effort to write and actually use them in your application. Yaaaay!

J.Ja