Title
Towards a Cost Model to Optimize User-Defined Functions in an ETL Workflow Based on User-Defined Performance Metrics
Authors
[ 1 ] Wydział Informatyki, Politechnika Poznańska | [ 2 ] Instytut Informatyki, Wydział Informatyki, Politechnika Poznańska | [ P ] employee
Scientific discipline (Law 2.0)
Year of publication
2019
Chapter type
chapter in monograph / paper
Publication language
english
Keywords
EN
- ETL workflow
- ETL execution optimization
- user-defined functions
- cost model
- parallelization
Abstract
EN Today’s ETL tools provide capabilities for developing custom code as user-defined functions (UDFs) to extend the expressiveness of standard ETL operators. However, a custom code of an UDF may execute inefficiently due to its poor implementation (e.g., due to the lack of using parallel processing or adequate data structures). In this paper we address the problem of the optimization of UDFs in data-intensive workflows and presented our approach to construct a cost model to determine the degree of parallelism for parallelizable UDFs.
Date of online publication
13.08.2019
Pages (from - to)
441 - 456
Ministry points / chapter
20
Ministry points / conference (CORE)
70
System created by Poznań University of Technology
and Poznan Supercomputing and Networking Center
Log in through eKonto to add to SIS