# Rex-Flink

**Repository Path**: mirrors_adjust/Rex-Flink

## Basic Information

- **Project Name**: Rex-Flink
- **Description**: automate Flink tasks with Rex
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 1
- **Forks**: 0
- **Created**: 2020-08-08
- **Last Updated**: 2025-12-02

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

=pod

=encoding UTF-8

=for :stopwords Ferenc Erki adjust GmbH CMDB CPAN Flink hadoop Hadoop libvirt scala VM YAML

=head1 NAME

Rex::Flink - automate Flink tasks with Rex

=head1 VERSION

version v0.5.0

=head1 SYNOPSIS

    # Rexfile
    use Rex::Flink;

    # CLI
    $ rex Flink:setup
    $ rex Flink:start
    $ rex Flink:stop

=head1 DESCRIPTION

This module automates Flink tasks with Rex.

B<Please note this is a work still heavily in progress, the interface might be subject to change before version 1.0.0.>

=head1 TASKS

=head2 Flink:setup

Sets Flink up from binary tarball. The installation directory can be controlled from CMDB via C<install_path>. Default is C<~/flink>.

Supported options:

=over 4

=item --hadoop

Hadoop version (default: C<27>)

=item --scala

Scala version (default: C<2.11>)

=item --version

Flink version (default: C<1.2.0>)

=back

=head2 Flink:configure

Configures Flink from a file, which is treated as a standard Rex template. The name of the file to be deployed can be controlled from CMDB via C<config_file>. Default is C<templates/flink-conf.yaml>.

=head2 Flink:start

Starts a Flink service.

Supported options:

=over 4

=item --service

Service name to start (default: C<jobmanager>)

=item --mode

Execution mode to start the service with (default: C<local>)

=back

=head2 Flink:stop

Stops a Flink service.

Supported options:

=over 4

=item --service

Service name to stop (default: C<jobmanager>)

=back

=head2 Flink:Pipeline:upload

Uploads a compiled pipeline to the remote machine from the local one. By default it uploads into C<~/flink/current/pipelines>.

Supported options:

=over 4

=item --file

Local filename of the pipeline to upload.

=back

=head2 Flink:Pipeline:run

Runs a pipeline on the remote machine.

Supported options:

=over 4

=item --file

Remote filename of the pipeline to run. Mandatory, without any default value.

=item --class

Class with the pipeline entry point.

=item --parallelism

Parallelism for the pipeline.

=item --options

Any extra options to be passed to the pipeline.

=back

=head1 DIAGNOSTICS

=over 4

=item *

C<No pipeline file specified>

=back

No pipeline file is specified to run with C<Flink:Pipeline:run> task.

=head1 CONFIGURATION AND ENVIRONMENT

This module can use configuration specified under the C<flink> node in the CMDB. Example YAML structure:

 flink:
     config_file: templates/flink-conf.yaml
     hadoop: 27
     install_path: ~/flink
     scala: 2.11
     version: 1.2.0

This module does not use any environment variables. See also L<TESTING>.

=head1 DEPENDENCIES

This module depends on the modules detailed below.

CPAN modules:

=over 4

=item *

Const::Fast

=item *

Rex

=back

=head1 INCOMPATIBILITIES

There are no known incompatibilities with other modules.

=head1 BUGS AND LIMITATIONS

There are no known bugs. Make sure they are reported.

=head1 TESTING

This module is capable to do testing with the help of the C<Rex::Test> module inside a VM via libvirt.

A URL pointing to a VM image to be used for the testing should be specified via the C<REX_FLINK_TEST_IMAGE> environment variable. For example:

 # CLI
 $ REX_FLINK_TEST_IMAGE='file:///var/lib/livirt/images/image.qcow2' rex Test:run
 $ REX_FLINK_TEST_IMAGE='https://domain.tld/image.qcow2' rex Test:run

=head1 AUTHOR

Ferenc Erki <ferki@adjust.com>

=head1 COPYRIGHT AND LICENSE

This software is copyright (c) 2017 by adjust GmbH.

This is free software; you can redistribute it and/or modify it under
the same terms as the Perl 5 programming language system itself.

=cut