Puppet Class: filebeat

Inherits:
filebeat::params
Defined in:
manifests/init.pp

Overview

This class installs the Elastic filebeat log shipper and helps manage which files are shipped

class { ‘filebeat’:

outputs => {
  'logstash' => {
    'hosts' => [
      'localhost:5044',
    ],
  },
},

}

proxy_address [String] Proxy server to use for downloading files

Parameters:

  • package_ensure (String) (defaults to: $filebeat::params::package_ensure)

    The ensure parameter for the filebeat package (default: present)

  • manage_package (Boolean) (defaults to: $filebeat::params::manage_package)

    Whether ot not to manage the installation of the package (default: true)

  • manage_repo (Boolean) (defaults to: $filebeat::params::manage_repo)

    Whether or not the upstream (elastic) repo should be configured or not (default: true)

  • manage_apt (Boolean) (defaults to: $filebeat::params::manage_apt)

    Whether or not the apt class should be explicitly called or not (default: true)

  • major_version (Enum['5','6', '7', '8']) (defaults to: $filebeat::params::major_version)

    The major version of Filebeat to be installed.

  • service_ensure (Variant[Boolean, Enum['stopped', 'running']]) (defaults to: $filebeat::params::service_ensure)

    The ensure parameter on the filebeat service (default: running)

  • service_enable (Boolean) (defaults to: $filebeat::params::service_enable)

    The enable parameter on the filebeat service (default: true)

  • repo_priority (Optional[Integer]) (defaults to: undef)

    Repository priority. yum and apt supported (default: undef)

  • service_provider (Optional[String]) (defaults to: $filebeat::params::service_provider)

    The provider parameter on the filebeat service (default: on RedHat based systems use redhat, otherwise undefined)

  • spool_size (Integer) (defaults to: $filebeat::params::spool_size)

    How large the spool should grow before being flushed to the network (default: 2048)

  • idle_timeout (String) (defaults to: $filebeat::params::idle_timeout)

    How often the spooler should be flushed even if spool size isn’t reached (default: 5s)

  • publish_async (Boolean) (defaults to: $filebeat::params::publish_async)

    If set to true filebeat will publish while preparing the next batch of lines to send (defualt: false)

  • config_dir (String) (defaults to: $filebeat::params::config_dir)

    The directory where inputs should be defined (default: /etc/filebeat/conf.d)

  • config_dir_mode (String[4,4]) (defaults to: $filebeat::params::config_dir_mode)

    The unix permissions mode set on the configuration directory (default: 0755)

  • config_dir_owner (Optional[String]) (defaults to: $filebeat::params::config_dir_owner)

    The owner of the configuration directory (default: root). Linux only.

  • config_dir_group (Optional[String]) (defaults to: $filebeat::params::config_dir_group)

    The group of the configuration directory (default: root). Linux only.

  • config_file (String) (defaults to: $filebeat::params::config_file)

    Where the configuration file managed by this module should be placed. If you think you might want to use this, read the [limitations](#using-config_file) first. Defaults to the location that filebeat expects for your operating system.

  • config_file_mode (String[4,4]) (defaults to: $filebeat::params::config_file_mode)

    The unix permissions mode set on configuration files (default: 0644)

  • config_file_owner (Optional[String]) (defaults to: $filebeat::params::config_file_owner)

    The owner of the configuration files, including inputs (default: root). Linux only.

  • config_file_group (Optional[String]) (defaults to: $filebeat::params::config_file_group)

    The group of the configuration files, including inputs (default: root). Linux only.

  • purge_conf_dir (Boolean) (defaults to: $filebeat::params::purge_conf_dir)

    Should files in the input configuration directory not managed by puppet be automatically purged

  • enable_conf_modules (Boolean) (defaults to: $filebeat::params::enable_conf_modules)

    Should filebeat.config.modules be enabled

  • modules_dir (String) (defaults to: $filebeat::params::modules_dir)

    The directory where module configurations should be defined (default: /etc/filebeat/modules.d)

  • http (Hash) (defaults to: $filebeat::params::http)

    A hash of the http section of configuration

  • cloud (Hash) (defaults to: $filebeat::params::cloud)

    Will be converted to YAML for the optional cloud of the configuration (see documentation, and above)

  • queue (Hash) (defaults to: $filebeat::params::queue)

    Will be converted to YAML for the optional queue of the configuration (see documentation, and above)

  • outputs (Hash) (defaults to: $filebeat::params::outputs)

    Will be converted to YAML for the required outputs section of the configuration (see documentation, and above)

  • shipper (Hash) (defaults to: $filebeat::params::shipper)

    Will be converted to YAML to create the optional shipper section of the filebeat config (see documentation)

  • logging (Hash) (defaults to: $filebeat::params::logging)

    Will be converted to YAML to create the optional logging section of the filebeat config (see documentation)

  • modules (Array) (defaults to: [])

    Will be converted to YAML to create the optional modules section of the filebeat config (see documentation)

  • conf_template (String) (defaults to: $filebeat::params::conf_template)

    The configuration template to use to generate the main filebeat.yml config file

  • download_url (Optional[Variant[Stdlib::HTTPUrl, Stdlib::HTTPSUrl]]) (defaults to: undef)

    The URL of the zip file that should be downloaded to install filebeat (windows only)

  • install_dir (Optional[String]) (defaults to: $filebeat::params::install_dir)

    Where filebeat should be installed (windows only)

  • tmp_dir (String) (defaults to: $filebeat::params::tmp_dir)

    Where filebeat should be temporarily downloaded to so it can be installed (windows only)

  • shutdown_timeout (String) (defaults to: $filebeat::params::shutdown_timeout)

    How long filebeat waits on shutdown for the publisher to finish sending events

  • beat_name (String) (defaults to: $filebeat::params::beat_name)

    The name of the beat shipper (default: FQDN)

  • tags (Array) (defaults to: $filebeat::params::tags)

    A list of tags that will be included with each published transaction

  • max_procs (Optional[Integer]) (defaults to: $filebeat::params::max_procs)

    The maximum number of CPUs that can be simultaneously used

  • fields (Hash) (defaults to: $filebeat::params::fields)

    Optional fields that should be added to each event output

  • fields_under_root (Boolean) (defaults to: $filebeat::params::fields_under_root)

    If set to true, custom fields are stored in the top level instead of under fields

  • disable_config_test (Boolean) (defaults to: $filebeat::params::disable_config_test)

    If set to true, configuration tests won’t be run on config files before writing them.

  • ssl (Hash) (defaults to: $filebeat::params::ssl)

    Optional fields set the ssl-configuration for input

  • processors (Array) (defaults to: [])

    Processors that will be added. Commonly used to create processors using hiera.

  • monitoring (Optional[Hash]) (defaults to: undef)

    The monitoring section of the configuration file.

  • inputs (Variant[Hash, Array]) (defaults to: {})

    or [Array] Inputs that will be created. Commonly used to create inputs using hiera

  • setup (Hash) (defaults to: {})

    setup that will be created. Commonly used to create setup using hiera

  • xpack (Optional[Hash]) (defaults to: $filebeat::params::xpack)

    Configuration items to export internal stats to a monitoring Elasticsearch cluster

  • extra_validate_options (Optional[String]) (defaults to: undef)

    Extra command line options to pass to the configuration validation command

  • autodiscover (Hash) (defaults to: $filebeat::params::autodiscover)

    Will be converted to YAML for the optional autodiscover section of the configuration (see documentation, and above)

  • overwrite_pipelines (Boolean) (defaults to: $filebeat::params::overwrite_pipelines)

    If set to true filebeat will overwrite (ingest) pipeline in Elasticsearch

  • run_options (Hash) (defaults to: $filebeat::params::run_options)
  • proxy_address (Optional[Variant[Stdlib::HTTPUrl, Stdlib::HTTPSUrl]]) (defaults to: undef)
  • filebeat_path (Stdlib::Absolutepath) (defaults to: $filebeat::params::filebeat_path)
  • queue_size (Integer) (defaults to: 4096)
  • registry_file (String) (defaults to: 'filebeat.yml')
  • systemd_beat_log_opts_override (Optional[String]) (defaults to: undef)
  • systemd_beat_log_opts_template (String) (defaults to: $filebeat::params::systemd_beat_log_opts_template)
  • systemd_override_dir (String) (defaults to: $filebeat::params::systemd_override_dir)
  • registry_path (Optional[String]) (defaults to: $filebeat::params::registry_path)
  • registry_file_permissions (Optional[String]) (defaults to: $filebeat::params::registry_file_permissions)
  • registry_flush (Optional[String]) (defaults to: $filebeat::params::registry_flush)


68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
# File 'manifests/init.pp', line 68

class filebeat (
  String  $package_ensure                                             = $filebeat::params::package_ensure,
  Boolean $manage_package                                             = $filebeat::params::manage_package,
  Boolean $manage_repo                                                = $filebeat::params::manage_repo,
  Boolean $manage_apt                                                 = $filebeat::params::manage_apt,
  Enum['5','6', '7', '8'] $major_version                              = $filebeat::params::major_version,
  Variant[Boolean, Enum['stopped', 'running']] $service_ensure        = $filebeat::params::service_ensure,
  Boolean $service_enable                                             = $filebeat::params::service_enable,
  Optional[String]  $service_provider                                 = $filebeat::params::service_provider,
  Optional[Integer] $repo_priority                                    = undef,
  Integer $spool_size                                                 = $filebeat::params::spool_size,
  String  $idle_timeout                                               = $filebeat::params::idle_timeout,
  Boolean $publish_async                                              = $filebeat::params::publish_async,
  String  $config_file                                                = $filebeat::params::config_file,
  Optional[String] $config_file_owner                                 = $filebeat::params::config_file_owner,
  Optional[String] $config_file_group                                 = $filebeat::params::config_file_group,
  String[4,4]  $config_dir_mode                                       = $filebeat::params::config_dir_mode,
  String  $config_dir                                                 = $filebeat::params::config_dir,
  String[4,4]  $config_file_mode                                      = $filebeat::params::config_file_mode,
  Optional[String] $config_dir_owner                                  = $filebeat::params::config_dir_owner,
  Optional[String] $config_dir_group                                  = $filebeat::params::config_dir_group,
  Boolean $purge_conf_dir                                             = $filebeat::params::purge_conf_dir,
  String  $modules_dir                                                = $filebeat::params::modules_dir,
  Boolean $enable_conf_modules                                        = $filebeat::params::enable_conf_modules,
  Hash    $http                                                       = $filebeat::params::http,
  Hash    $cloud                                                      = $filebeat::params::cloud,
  Hash    $queue                                                      = $filebeat::params::queue,
  Hash    $outputs                                                    = $filebeat::params::outputs,
  Hash    $shipper                                                    = $filebeat::params::shipper,
  Hash    $logging                                                    = $filebeat::params::logging,
  Hash    $run_options                                                = $filebeat::params::run_options,
  String  $conf_template                                              = $filebeat::params::conf_template,
  Optional[Variant[Stdlib::HTTPUrl, Stdlib::HTTPSUrl]] $download_url  = undef, # lint:ignore:140chars
  Optional[String]  $install_dir                                      = $filebeat::params::install_dir,
  String  $tmp_dir                                                    = $filebeat::params::tmp_dir,
  String  $shutdown_timeout                                           = $filebeat::params::shutdown_timeout,
  String  $beat_name                                                  = $filebeat::params::beat_name,
  Array   $tags                                                       = $filebeat::params::tags,
  Optional[Integer] $max_procs                                        = $filebeat::params::max_procs,
  Hash $fields                                                        = $filebeat::params::fields,
  Boolean $fields_under_root                                          = $filebeat::params::fields_under_root,
  Hash $ssl                                                           = $filebeat::params::ssl,
  Boolean $disable_config_test                                        = $filebeat::params::disable_config_test,
  Array   $processors                                                 = [],
  Optional[Hash]  $monitoring                                         = undef,
  Variant[Hash, Array] $inputs                                        = {},
  Hash    $setup                                                      = {},
  Array   $modules                                                    = [],
  Optional[Variant[Stdlib::HTTPUrl, Stdlib::HTTPSUrl]] $proxy_address = undef, # lint:ignore:140chars
  Stdlib::Absolutepath $filebeat_path                                 = $filebeat::params::filebeat_path,
  Optional[Hash] $xpack                                               = $filebeat::params::xpack,

  Integer $queue_size                                                 = 4096,
  String $registry_file                                               = 'filebeat.yml',

  Optional[String] $systemd_beat_log_opts_override                    = undef,
  String $systemd_beat_log_opts_template                              = $filebeat::params::systemd_beat_log_opts_template,
  String $systemd_override_dir                                        = $filebeat::params::systemd_override_dir,
  Optional[String] $extra_validate_options                            = undef,
  Hash $autodiscover                                                  = $filebeat::params::autodiscover,
  Optional[String] $registry_path                                     = $filebeat::params::registry_path,
  Optional[String] $registry_file_permissions                         = $filebeat::params::registry_file_permissions,
  Optional[String] $registry_flush                                    = $filebeat::params::registry_flush,
  Boolean $overwrite_pipelines                                        = $filebeat::params::overwrite_pipelines,

) inherits filebeat::params {
  include stdlib

  $real_download_url = $download_url ? {
    undef   => "https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-${package_ensure}-windows-${filebeat::params::url_arch}.zip",
    default => $download_url,
  }

  if $config_file != $filebeat::params::config_file {
    warning('You\'ve specified a non-standard config_file location - filebeat may fail to start unless you\'re doing something to fix this')
  }

  if $package_ensure == 'absent' {
    $alternate_ensure = 'absent'
    $real_service_ensure = 'stopped'
    $file_ensure = 'absent'
    $directory_ensure = 'absent'
    $real_service_enable = false
  } else {
    $alternate_ensure = 'present'
    $file_ensure = 'file'
    $directory_ensure = 'directory'
    $real_service_ensure = $service_ensure
    $real_service_enable = $service_enable
  }

  # If we're removing filebeat, do things in a different order to make sure
  # we remove as much as possible
  if $package_ensure == 'absent' {
    anchor { 'filebeat::begin': }
    -> class { 'filebeat::config': }
    -> class { 'filebeat::install': }
    -> class { 'filebeat::service': }
    -> anchor { 'filebeat::end': }
  } else {
    if !$manage_package {
      anchor { 'filebeat::begin': }
      -> class { 'filebeat::config': }
      -> class { 'filebeat::service': }
      -> anchor { 'filebeat::end': }
    } else {
      anchor { 'filebeat::begin': }
      -> class { 'filebeat::install': }
      -> class { 'filebeat::config': }
      -> class { 'filebeat::service': }
      -> anchor { 'filebeat::end': }
    }
  }

  if $package_ensure != 'absent' {
    if !empty($inputs) {
      if $inputs =~ Array {
        create_resources('filebeat::input', { 'inputs' => { pure_array => true } })
      } else {
        create_resources('filebeat::input', $inputs)
      }
    }
  }
}