Yes because there is a layer of abstraction between the development process and the output (today we would think it insane to manually write code in a text editor) and no because the abstraction layer is not high enough (meaning that still very task orientated).Įnter the realm of automation technologies. Is the process of the ETL development using an IDE similar to that of the modeller (and therefore argued as automated)? I think yes and no. The developer links together components to perform some action or task and each task/package must be manually developed and configured (for example, you would write 10 tasks to extract 10 tables into a staging area). Sure, there’s a tool that is used (eg SSIS, Kettle, Talend) which acts as an IDE however, the development process is still labour intensive. In contrast, when it comes to ETL development, most processes manual. We expect the modeller to use a tool to manage and maintain the modelling process (eg ERwin, DeZign) and would think it unprofessional and prone to error if components were hand coded manually. Data is extracted into a staging area, transformed into a warehouse, deployed to a presentation server/layer and delivered via a user interface (reporting/analytic layer). Timextender history load series#Regardless of the methodology used, most employ a series of processes and silos that do not change from implementation to implementation. I suggest that the conceptual design is driven by understood practices#1 – to the extent of a cookie cutter template. The last stage (ETL development) requires the largest effort on a project and has been documented to require more than 60% of total effort. In traditional development, the system is designed is designed at a conceptual level, a data modeller designs and builds the relational framework (the schema) and then ETL developers populate it. the system must be capable of changing requirement.the ‘delivery’ requirements of the system will change.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |