Transformers

Transformers change the list of extracted items in some way, they can be chained multiple times to obtain the needed result.

ColumnRenameTransformer

This transformer allows the renaming of columns (source column to new name column for the destination)

Parameters

$job->transform('App\Etl\Transformers\ColumnRenameTransformer', [
     'columns' => [
         'C1262'  => 'id',
         'C1263'  => 'name',
         'C1274'  => 'username',
         'C1269'  => 'email',
         'C1752'  => 'acronym',
         'C1276'  => 'locale',
         'C1267'  => 'company_phone_number',
         'C1267'  => 'national_id_number',
         'C21174' => 'number'
    ]
]);

IncludeOnlyColumnsTransformer

This transformer excludes all columns not in the provided list

Parameters

$job->transform('App\Etl\Transformers\IncludeOnlyColumnsTransformer', [
    'columns' => [
         'id', 
         'name', 
         'username',
         'email',
         'acronym',
         'locale',
         'company_phone_number',
         'national_id_number',
         'number'
    ]
]);

EncryptColumnsTransformer

This transformer encrypts the value of the column (Using laravel's Crypt::encrypt() function)

Parameters

$job->transform('App\Etl\Transformers\EncryptColumnsTransformer', [
    'columns' => [
        'national_id_number'
    ],
    'key' => base64_decode('base64:DuJBVxAMTPzC5YWNpAniTTL8gxOe04CxSxvrfgcx0qw=')
]);

AddColumnsStaticValueTransformer

This transformer adds a new column with a static value (value will be repeated for all rows)

Parameters

$job->transform('App\Etl\Transformers\AddColumnsStaticValueTransformer', [
    'new_columns' => [
        'password' => \Hash::make('wemake'),
        'system_reserved_user' => 0,
        'status_id' => 1,
        'sync_status_with_external_auth_provider' => 0,
        'created_at' => $now,
        'updated_at' => $now
    ]
]);

AddColumnsCallbackValueTransformer

This transformer adds a new column with a value returned by a given callback

$job->transform('App\Etl\Transformers\AddColumnsCallbackValueTransformer', [
    'new_columns' => [
         'email' => function($row) {
              return $row['username'] . '@wemake.pt';
         },
         'acronym' => function($row) { 
              return (!empty($row['acronym'])) ? $row['acronym'] : $row['username'];
         },
         'locale' => function($row) { 
              $locale = strtolower($row['locale']);
              return (in_array($locale, ['pt', 'en'])) ? $locale : 'pt'; 
         }
     ]
]);

TransformColumnAccordingToTableSequenceTransformer

This transformer will update the values of any given column to use the diff stored in the sequencer.

$job
    ->transform('App\Etl\Transformers\DropRowsIfCallbackFailTransformer', [
        'columns' => [
            'user_id' => 't00_users'
        ]
    ])

DropRowsIfCallbackFailTransformer

This transformer drops rows if the callback returns false

$job
    ->transform('App\Etl\Transformers\DropRowsIfCallbackFailTransformer', [
        'callback' => function($item) {
            return $item['user_id'] > 0;
        }
    ])

RemakeItemsViaCallbackTransformer

This transformer uses a callback to completelly transform the items list into a new items list (only what the callback returns will be passed on, the old items will be discarded). The callback must return an array of items, so there is the potential for one old row to become many new ones.

$job->transform('App\Etl\Transformers\RemakeItemsViaCallbackTransformer', [
                'callback' => function($item) use ($document_actions) {
                    $new_items = collect([]);

                    if($item['v9_permissions__alter_constitution']) {
                        $new_items->push([
                            'family_id' => $item['family_id'],
                            'action_id' => $document_actions['change-constitution']
                        ]);
                    }

                    if($item['v9_permissions__alter_accesses']) {
                        $new_items->push([
                            'family_id' => $item['family_id'],
                            'action_id' => $document_actions['change-access']
                        ]);
                    }

                    if($item['v9_permissions__alter_workflow']) {
                        $new_items->push([
                            'family_id' => $item['family_id'],
                            'action_id' => $document_actions['change-workflow']
                        ]);
                    }

                    if($item['v9_permissions__alter_notification_definitions']) {
                        $new_items->push([
                            'family_id' => $item['family_id'],
                            'action_id' => $document_actions['change-definition-notifications']
                        ]);
                    }

                    return $new_items->values()->toArray();
                }
            ]);

GroupByTransformer and MergeByTransformer

These two transformers work together to "collapse" and "uncollapse" the items list by grouping it by a selected column, you also have the opurtunity to isolate a group of columns in the group by.

GroupByTransformer

MergeByTransformer

CopyFileSourceToDestinationTransformer

This transformer will copy files referenced by items (this is important) from a source folder to a destination folder. You may also use a renaming callback to rename the files being copied.

CalculateBaumNodesFromParentIdTransformer

This transformer receives items with parent_id and id (these columns must exist) and calculates the values for lft, rgt and depth, used in v10 by Baum/Node for preserving a tree structure in database.