Almost a year back based on an ASUG event I have written in my blog Data Management in Modern Enterprise focusing on harnessing the power of in-memory platforms in data migration issues. I have tested this in my lab to prove to myself the viability of this usecase. Test data is always the test data.
Soon after this I got onto an engagement on one of the biggest retailers in Canada on SAP HANA. As an EA / Program Manager my focus has been in supporting the HANA appliance (very large scale-out solution support 18TB of data in memory, considered as one of the largest scale-out solutions in the world). In this process I had been part of architectural review sessions focusing on performance aspects of loading historical data into SAP HANA.
Without going into too many details, to summarize the findings from performance perspective
a) Data loading via SLT is good (after doing some network fine tuning)
b) Historical data loading via SAP BO (covering lot of transformations ) was very slow. This
data is being migrated from Teradata onto SAP HANA. It was so slow that to migrate full data
it takes months
Upon further analysis it was found that most of part of the data load time was being spent in SAP BO. At this point of time One of my recommendations was to do ELT ( figure 2, in the previous blog ) instead of ETL. So as initial attempt we moved all the integration driven transformations (cross lookups, parent child relationships) over to SAP HANA as internal procedures.
Since then historical data loads were never been a bottle neck and what was supposed to take months before, now takes weeks ( with right # of sessions from Teradata). So.. to conclude ELT is very much a practical and needful approach in processing big data to harness the power of the modern day in-memory platforms like SAP HANA.
