In data warehouse projects, integrating SAP data, one particular question usually pops up a couple of months into the project:
“What about SAP Long Text?”
Long Text (also referred to as SAPscript texts or text objects) are the containers attached to SAP ERP objects to accommodate long text in SAP systems. Users can put free text and even apply formatting and they are not blocked by common database or application restrictions. Users put long text on common SAP objects like sales orders, materials, or notifications. SAP created this functionality because old database systems had text columns limited to around 255 characters. The “new” database systems don’t have this restriction, but Long Text remained anyway.
SAP stores the Long Text in a particular format within a special structure in the application. The table STXL, for instance, saves the Long Text details in a blob similar to SAP’s cluster tables. This structure makes it hard for traditional ETL and more current data acquisition methods to get the data in a transparent format on a target system and requires a lot of SAP expertise.
“So what about Long Text?”
Back in the day, my first reaction was, “what value does this type of unstructured data add?” How would the Long Text fit within the defined structure of the data warehouse/data mart? This data was not harmonized and would need a lot of sanitizing before it could be of use. Further, seeing the effort required for integration, I gave the advice to use other fields to store information. For example, if a sales department was managing the sales order status in the Long Text, wouldn’t it be much better to use/configure the sales order status field?
So much for good advice…
SAP users liked the freedom the Long Text provided and using common fields to store the same information could require a long change process. In the end, all implementations, without exception, added Long Text structures to their extraction and data warehouses with experts’ help on the integration side. The requirement and use cases for this kind of data have lately only grown as analytical methods nowadays are more equipped to handle unstructured data. Text and ML analysis on tickets or maintenance orders can offer new insights.
Ok, so Long Text integration will remain a normal challenge?
Long Text integration has become easier this year because HVR can replicate Long Text directly from SAP ECC or S/4HANA. The blob containing the Long Text in STXL will be made transparent during the replication process without much additional effort. As a result, HVR users can source near real-time Long Text on all SAP objects with the same software and method as transparent or other cluster and pool tables.
HVR supports a significant number of targets from traditional databases (e.g., Oracle, SQL Server), data warehouse systems (Snowflake, Teradata), or file-based solutions like ADLS or KAFKA. This enables companies to use their SAP Long Text in many different target systems without much latency. Of course, the use case is dependent on how an organization uses Long Text. Still, the effort to integrate text, for example, on sales orders (for status analysis) or text on repair orders (for failure analysis), is no longer an argument for leaving this data in the SAP source.
If you’re interested in learning more about HVR’s SAP data replication capabilities, feel free to contact us or check out these additional resources: