Building an Order/Invoice System in OutSystems — Part 1

A quick history: about five years ago, I joined an OutSystems project, and my first task was to build an invoice system and a way to make credit card and ACH purchases. At the time, I had done an invoice system or two in the past, but not in OutSystems, so I had a fair idea of what it would take to get it done. The good news is, it was very easy to do in OutSystems! The bad news is, I did not know nearly as much as I needed to know to build a robust invoice and order system, and I learned some painful lessons along the way.

Before we get into the technical implementation, we are going to look at the traits of a good order/invoice system.

Completely Separate Payments Code/Data Model

One thing you never want to do is to tightly couple your payment processor to your order/invoice system. If you need to handle multiple countries this is almost a guarantee, and some countries do not have a single payment processor that can do everything you need. And of course, you may need to change payment processors in the future!

You will want to completely isolate the code and data that is used by payments, and present a simple set of Actions to the rest of the system. Even better, once you have done that, hide the specific implementation from the rest of the system behind Actions that make life easy on the implementation.

For example, your payment processor (let’s call them “Super Pay”) has a function for “Make Credit Card Payment”. In your SuperPay module you make a “SuperPay_CreditCardPayment” Action that accepts the parameters needed for payment. Then in your Payments module, have a “MakePayment” Action that accepts the needed information (credit card number, expiration date, etc.), and determines which processor to use, and calls the appropriate Actions to make it all work (in this case, SuperPay_CreditCardPayment). This may seem like a ton of extra effort with just one payment processor, but as soon as you add a second, you will be glad you did this.

Be Aware of PCI

PCI compliance is a big ball of headaches but it is not nearly as tricky as people make it out to be either. The basic rule of thumb with PCI compliance: you must encrypt or not store any/all personally identifiable information (PII, data that can be used to identify who someone is, how to contact, etc.) or sensitive information that can be used to make transactions on their behalf (like their credit card number). For the precise details of what must be encrypted and/or not stored, you will need to do some research on your own to learn what specific PCI rules apply to your situation. But you can summarize the basics as “HTTPS everywhere” and “encrypt at rest or do not store customer and payment method related data”.

Be Aware of Other Compliance Issues

PCI is not the only compliance challenge out there. The EU zone, for example, has a pile of regulations around direct debits to bank accounts. The UK has even more strict rules than the EU. You will need to pay very close attention to the regulatory environment of the countries you will be working in. This is another reason that it is important to divide the code up into very discrete modules and actions, otherwise it is hard to handle the regulatory aspects as you grow.

Three Levels of Payment Integration

With PCI looming over your head, how do you make transactions? Payment processors typically offer three levels of payment integration:

• Redirect the user to their hosted page to checkout. Usually your system sends over enough information about the order for them to provide a payment page, the user makes the payment, then gets sent back to your system with a code indicating that payment was made. Pros: easy integration, less worries about PCI. Cons: you lose control over the UI and user experience of the checkout process, it usually “feels” bad to the user and makes your system look like you have a low end, cheap system (PayPal is probably the only exception to this), cannot automate payments beyond some system for setting up recurring payments.
• Put a hosted payment page inside an iframe on your checkout page. The payment provider will have a smaller page that will look at home inside your site, to some extent, and at least the URL on the browser is still in your domain. Pros: easy integration, less worries about PCI, a bit more of a smooth feel. Cons: you still do not have much control over the user experience, cannot integrate automated payments beyond their recurring payments system.
• Integrate through some sort of API. Pros: 100% control over the user experience and automated payments.

If you are planning to use one of their hosted payment page solutions, have read their documentation, and feel fully comfortable that this meets your needs, you will still have the challenge of building the ordering and invoicing system, but you have saved yourself the hassle of handling the checkout system.

I really do not recommend this route for any project that operates at-scale, has more than a few users, etc. The end user always gets the feeling that something is wrong with their checkout experience.

If you have any kind of automated billing needs, unless it is a very simple, never-changing recurring charge, I have found the their systems for recurring billing to not be good enough. The basic problem is that there is no way to adjust any single scheduled payment, so if a customer wants to go on hold, needs to change the date of their next payment, has a one-time discount, etc. you need to cancel the entire payment schedule and make a new one. As soon as you want to support something like this, you need to be managing automated payments yourself and having your scheduled payment system integrate through their API.

“Accumulate” Data to Tell A Story

The first few times I made invoicing data models, I made them too simple. There was no good way to “rewind” the changes to an invoice or a line item, especially with regards to discounts and manual price overrides. The solution to this problem is to build a data model that “accumulates” and then presents the “final” numbers to the user. When we get to looking at the details of the data model for invoices and line items you will see exactly what I mean.

Always Store Data “As It Was”

Imagine that you are building a message forum. Every time a user posts a message, would you copy that user’s profile information to the message and use that copy when displaying the message to show the poster’s name and picture? No, of course not! This would be very wasteful of the database’s storage space, and if the user ever changed their name, you would either need to change all of this copied data, or you would end up having the older message display the old name, which would be really confusing. Instead, each message record has a foreign key to the user that posted it, and at the time of display you get that user’s profile and show their current data.

An order/invoice system is exactly the opposite. You must maintain the integrity of the data “as it was”. Imagine this scenario: on June 8th, a user buys 5 widgets for $5 each. On June 10th, the price of a widget changes to $7. On June 14th, the user returns the widgets. If you had linked the order to the widget record and used that linked record to handle the return, you would have refunded the user $7 for the widgets they paid $5 for. Oops!

There are two things you should be doing here (and you may want to consider both):

1. Have the product data model follow an archive pattern for all data changes. Whenever a product is changed, never update the existing row, always create a new row with a new ID, and mark the previous row as “archived”. This way, the product that the line items are tied to always show the product at the time of the order when they view description, price, etc.
2. Make sure the data model for the invoice and line items stores the information in them as it is used, separately from their backing product information. For example, when a user adds a product to the invoice, it copies the current price for the product to the line item row, so no matter what happens to the product, the line item knows what the price was at the time of the order.

Note what both of these techniques have in common! The first is simply mandatory. You never want to have a situation where someone says “but the product description said…” and you cannot verify it. The second is a backup plan for the first, and makes invoice and line item changes much easier to deal with, and I highly recommend it.

The next article in this series will go into data model specifics around the invoices and line items.

J.Ja