Move Fields from package libbeat/common to libbeat/mapping. 11198

For "metricbeat style" generated custom beats, the mage target GoTestIntegration has changed to GoIntegTest and GoTestUnit has changed to GoUnitTest. 13341

Build docker and kubernetes features only on supported platforms. 13509

Need to register new processors to be used in the JS processor in their init functions. 13509

The custom beat generator now uses mage instead of python, mage GenerateCustomBeat can be used to create a new beat, and mage vendorUpdate to update the vendored libbeat in a custom beat. 13610

Altered all remaining uses of mapval to use the renamed and enhanced version: go-lookslike instead, which is a separate project. The mapval tree is now gone. 14165

Move light modules to OSS. 14369

Deprecate test flags, generate and update_expected, in favor of data. 15292

Python 3 is required now to run python tests and tools. 14798

The type memqueue.Broker is no longer exported; instead of memqueue.NewBroker, call memqueue.NewQueue (which provides the same public interface). 16667

The disk spool types spool.Spool and spool.Settings have been renamed to the internal types spool.diskSpool and spool.settings. 16693

queue.Eventer has been renamed to queue.ACKListener 16691

Require logger as first parameter for outputs.transport.transport#ProxyDialer and outputs.elasticsearch.client#BulkReadItemStatus. 16761

The libbeat/outputs/transport package has been moved to libbeat/common/transport. 16734

The libbeat/outputs/tls.go file has been removed. All exported symbols in that file (libbeat/outputs.) are now available as libbeat/common/tlscommon.. 16734

The newly generated Beats are using go modules to manage dependencies. 16288

Extract Elasticsearch client logic from outputs/elasticsearch package into new esclientleg package. 16150

Rename queue.BufferConfig.Events to queue.BufferConfig.MaxEvents. 17622

Remove queue.Feature and replace queue.RegisterType with queue.RegisterQueueType. 17666

Introduce APM libbeat instrumentation. Publish method on Client interface now takes a Context as first argument. 17938

The way configuration files are generated has changed to make it easier to customize parts of the config without requiring changes to libbeat config templates. Generation is now fully based on Go text/template and no longer uses file concatenation to generate the config. Your magefile.go will require a change to adapt the devtool API. See the pull request for more details. 18148

The Elasticsearch client settings expect the API key to be raw (not base64-encoded). 18939 18945

management.ConfigManager has been renamed to management.Manager. 19114

UpdateStatus has been added to the management.Manager interface. 19114

Remove common.MapStrPointer parameter from cfgfile.Runnerfactory interface. 19135

Replace ACKCount, ACKEvents, and ACKLastEvent callbacks with ACKHandler and interface in beat.ClientConfig. 19632

Remove global ACK handler support via SetACKHandler from publisher pipeline. 19632

Make implementing Close required for reader.Reader interfaces. 20455

Stop using mage:import in community beats. This was ignoring the vendorized beats directory for some mage targets, using the code available in GOPATH, this causes inconsistencies and compilation problems if the version of the code in the GOPATH is different to the vendored one. Use of mage:import will continue to be unsupported in custom beats till beats is migrated to go modules, or mage supports vendored dependencies. 13998 14162

Metricbeat module builders call host parser only once when instantiating light modules. 20149

Add configuration for APM instrumentation and expose the tracer trough the Beat object. 17938

Make the behavior of clientWorker and netClientWorker consistent when error is returned from publisher pipeline

Metricset generator generates beta modules by default now. 10657

The beat.Event accessor methods now support @metadata keys. 10761

Assertion for documented fields in tests fails if any of the fields in the tested event is documented as an alias. 10921

Support for Logger in the Metricset base instance. 11106

Filebeat modules can now use ingest pipelines in YAML format. 11209

Prometheus helper for metricbeat contains now Namespace field for prometheus.MetricsMappings 11424

Update Jinja2 version to 2.10.1. 11817

Reduce idxmgmt.Supporter interface and rework export commands to reuse logic. 11777,https://github.com/elastic/beats/pull/12065[12065],https://github.com/elastic/beats/pull/12067[12067],https://github.com/elastic/beats/pull/12160[12160]

Update urllib3 version to 1.24.2 11930

Add libbeat/common/cleanup package. 12134

New helper to check for leaked goroutines on tests. 12106

Only Load minimal template if no fields are provided. 12103

Add new option IgnoreAllErrors to libbeat.common.schema for skipping fields that failed while converting. 12089

Deprecate setup cmds for template and ilm-policy. Add new setup cmd for index-management. 12132

Use the go-lookslike library for testing in heartbeat. Eventually the mapval package will be replaced with it. 12540

New ReporterV2 interfaces that can receive a context on Fetch(ctx, reporter), or Run(ctx, reporter). 11981

Generate configuration from mage for all Beats. 12618

Add ClientFactory to TCP input source to add SplitFunc/NetworkFuncs per client. 8543

Introduce beat.OutputChooses publisher mode. 12996

Ensure that beat.Processor, beat.ProcessorList, and processors.ProcessorList are compatible and can be composed more easily. 12996

Add support to close beat.Client via beat.CloseRef (a subset of context.Context). 13031

Add checks for types and formats used in fields definitions in fields.yml files. 13188

Makefile included in generator copies files from beats repository using git archive instead of cp. 13193

Strip debug symbols from binaries to reduce binary sizes. 12768

Compare event by event in testadata framework to avoid sorting problems 13747

Added a default_field option to fields in fields.yml to offer a way to exclude fields from the default_field list. 14262 14341

supported-versions.yml can be used in metricbeat python system tests to obtain the build args for docker compose builds. 14520

Fix dropped errors in the tests for the metricbeat Azure module. 13773

New mage target for Functionbeat: generate pkg folder to make manager easier. 15880

Add support for MODULE environment variable in mage goIntegTest in metricbeat to run integration tests for a single module. 17147

Add support for a TEST_TAGS environment variable to add tags for tests selection following go build tags semantics, this environment variable is used by mage test targets to add build tags. Python tests can also be tagged with a decorator (@beat.tag('sometag')). 16937 17075

Add fields validation for histogram subfields. 17759

Add IP* fields to fields.yml generator script in Filebeat. 17998 18256

Events intended for the Elasticsearch output can now take an op_type metadata field of type events.OpType or string to indicate the op_type to use for bulk indexing. 12606

Remove vendor folder from repository. 18655

Added SQL helper that can be used from any Metricbeat module 18955

Update Go version to 1.14.4. 19753

Update Go version to 1.14.7. 20508

Add packaging for docker image based on UBI minimal 8. 20576