Extractors

Version Date Notes By
0.2 2018-06-01 New extractors ROB
0.1 2018-01-23 Initial release ROB

Extractors retrieve the information to be processed by the Job.

LaravelTableExtractor

This extractor uses the laravel query builder to extract values from a table / connection. The query may be adjusted via a callback.

Parameters

Name Type Description
connection String The database connection to extract from
query_callback Callback($query) the callback will receive the query builder object to be used for extraction, allowing for changes to the query. 
$job->extract('App\Etl\Extractors\LaravelTableExtractor', 't03w004', [
    'connection' => 'source_db',
    'query_callback' => function($query) {
        $query->whereIn('C1262', [18, 19, 20]);
    }
]);

DirectTableExtractor

This extractor allows for the direct "extraction" of a provided array, to start the job with a fixed set of data.

Parameters

Name Type Description
items Array The list of items to "extract" 
$job
->extract('App\Etl\Extractors\DirectTableExtractor', 't03w027', [
    'items' => [
        [
            'id'        => 1,
            'name'      => 'Documentos',
            'locale'    => 'pt',
            'status_id' => 1
        ],[
            'id'        => 2,
            'name'      => 'Impressos',
            'locale'    => 'pt',
            'status_id' => 1
        ],[
            'id'        => 3,
            'name'      => 'Registos',
            'locale'    => 'pt',
            'status_id' => 1
        ]
    ]
]);

CsvExtractor

This extractor allows for extracting of data from a .csv file.

Parameters

Name Type Description
columns Array Indicates which column goes under which column name (for easier later treatment)
uses_quotes Boolean Indicates whether the file being extracted uses quotation marks (default false)
separator String Indicates the separator character being used 
$job
    ->extract('App\Etl\Extractors\CsvExtractor', '~/example.csv', [
        'columns'   => [
            // order - new name
            0 => 'table_name',
            1 => 'relatable_column_name',
            2 => 'relatable_column_id',
            3 => 'translation'
        ],
        // define separator character
        'separator' => ';'
    ])

JsonExtractor

This extractor allows for the extracting of data from a json source file (eg: .json)

Parameters

Name Type Description
columns Array define which of the json elements to import as colums for the data stream.