Commit 1f0c7962 authored by Walter Heck's avatar Walter Heck

Updates to documentation for 2.0.0

fixes #8
parent 82b78ba3
# Prerequisites
* Make sure you have read <https://gitlab.olindata.com/opstheater/opstheater-docs/blob/master/1.7/deployment/scenario.md>
* Make sure you have read <https://gitlab.olindata.com/opstheater/opstheater-docs/blob/master/2.0/deployment/scenario.md>
* Make sure you have installed the following on your host machine:
* virtualbox (<http://virtualbox.org>)
......@@ -26,54 +26,54 @@
* Clone the opstheater repository on your host machine:
```
git clone git@gitlab.olindata.com:opstheater/opstheater.git
```
```
git clone git@gitlab.olindata.com:opstheater/opstheater.git
```
* Switch to the vagrant environment directory
```
cd opstheater/deploy/vagrant-oscar
```
```
cd opstheater/deploy/vagrant-oscar
```
* All the deployment files and configurations are in the `opstheater/deploy/vagrant-oscar` directory.
* Bring up the puppet master (This will take a looong time!) vm and log into it:
For Puppet Enterprise:
```
vagrant up master
vagrant ssh master
sudo su -
```
For Open Source Puppet:
```
vagrant up foss-master
vagrant ssh foss-master
sudo su -
```
At the very end of that process the `vagrant up` output displays the login details for foreman:
```
==> foss-master: Notice: Applied catalog in 0.25 seconds
==> foss-master: ==> Foreman URL: http://master.opstheater.vm:3000
==> foss-master: ==> Login credentials: admin / zndxYuJbDzasVCpv
```
Don’t forget to add master.opstheater.vm to your hosts file.
For Puppet Enterprise:
```
vagrant up master
vagrant ssh master
sudo su -
```
For Open Source Puppet:
```
vagrant up foss-master
vagrant ssh foss-master
sudo su -
```
At the very end of that process the `vagrant up` output displays the login details for foreman:
```
==> foss-master: Notice: Applied catalog in 0.25 seconds
==> foss-master: ==> Foreman URL: http://master.opstheater.vm:3000
==> foss-master: ==> Login credentials: admin / zndxYuJbDzasVCpv
```
Don’t forget to add master.opstheater.vm to your hosts file.
* Now bring up one or more machines with oscar on your host machine:
```
vagrant up mysql
vagrant up icinga2
vagrant up gitlab
vagrant up elasticsearch
```
```
vagrant up mysql
vagrant up icinga2
vagrant up gitlab
vagrant up elasticsearch
```
to see which machines are available, simply do an `vagrant status`.
to see which machines are available, simply do an `vagrant status`.
NOTE: you might have to run puppet a number of times on each node to make sure it fully puppetises itself. Some of the initial runs might have errors but they will go away as the puppet runs on all servers start happening.
NOTE: you might have to run puppet a number of times on each node to make sure it fully puppetises itself. Some of the initial runs might have errors but they will go away as the puppet runs on all servers start happening.
* You might also want to install r10k or puppet-librarian on your host machine and install the modules the opstheater repo uses on your host so you can browse through them.
TODO: Add instructions for r10k setup
TODO: Add instructions for r10k setup
......@@ -9,46 +9,46 @@ TODO: this is a work in progress, it is not complete and might not be fully up t
1.1 which tools do you want to use of the OpsTheater stack
* Puppet? Open Source or Enterprise?
* The foreman?
* Grafana, Kibana, elasticsearch?
* Icinga?
* Gitlab, Mattermost?
* Puppet? Open Source or Enterprise?
* The foreman?
* Grafana, Kibana, elasticsearch?
* Icinga?
* Gitlab, Mattermost?
1.2 hostnames
Recommended…
Recommended…
* puppet.example.com (puppetmaster / foreman (if foss))
* icinga.example.com (icinga + icinga web)
* elastic.example.com (kibana / logstash / elasticsearch / influxdb / grafana)
* gitlab.example.com (gitlab / mattermost)
* mysql.example.com (mysql)
* puppet.example.com (puppetmaster / foreman (if foss))
* icinga.example.com (icinga + icinga web / influxdb)
* elastic.example.com (kibana / logstash / elasticsearch / grafana)
* gitlab.example.com (gitlab / mattermost)
* mysql.example.com (mysql)
1.3 resource allocation (either the default scheme or custom if need be)
1.4 HTTPS Yes or No?
If yes, need certificates for above hostnames
If yes, need certificates for above hostnames
1.5 users (name, email address), groups
We need to create a sheet of sorts to collect what types of users and what types of access to be given eg:
* Foreman users
* Puppet users
* Shell users (admin access)
* Gitlab / Mattermost users
* Icinga users
We need to create a sheet of sorts to collect what types of users and what types of access to be given eg:
* Foreman users
* Puppet users
* Shell users (admin access)
* Gitlab / Mattermost users
* Icinga users
1.6 SMTP Relay information for our stack being able to send emails
* SMTP Server Hostname
* SMTP Server Port
* Authenticated? If yes...
* username:
* Password:
* Uses TLS?
* Uses StartTLS?
* SMTP Server Hostname
* SMTP Server Port
* Authenticated? If yes...
* username:
* Password:
* Uses TLS?
* Uses StartTLS?
1.7 any custom requirements
......
......@@ -2,22 +2,22 @@
1. Configure repo and install puppet.
```
/bin/yum install -y epel-release http://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm
/bin/yum install -y puppet-agent
/opt/puppetlabs/bin/puppet config set --section main server puppet.olindata.com
```
```
/bin/yum install -y epel-release http://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm
/bin/yum install -y puppet-agent
/opt/puppetlabs/bin/puppet config set --section main server puppet.olindata.com
```
2. Configure/adjust Master
Ensure correct filename yaml file exists in below locations:
Ensure correct filename yaml file exists in below locations:
```
/etc/puppetlabs/code/environments/production/hieradata/10.fqdn
```
Logstash settings needs to be checked for public vs private ip
```
/etc/puppetlabs/code/environments/production/hieradata/10.fqdn
```
Logstash settings needs to be checked for public vs private ip
Ref: - "elastic.olindata.com:9200"
Ref: - "elastic.olindata.com:9200"
3. wget needs to be installed before running puppet agent -t.
......@@ -25,11 +25,11 @@
5. update site.pp as bellow:
```
node 'elastic.olindata.com' {
include opstheater::role::elastic::server
include opstheater::role::logstash::server
}
```
```
node 'elastic.olindata.com' {
include opstheater::role::elastic::server
include opstheater::role::logstash::server
}
```
6. Run Puppet agent.
This diff is collapsed.
......@@ -2,38 +2,38 @@
1. Configure repo and install puppet.
```
/bin/yum install -y epel-release http://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm
/bin/yum install -y puppet-agent
/opt/puppetlabs/bin/puppet config set --section main server puppet.olindata.com
```
```
/bin/yum install -y epel-release http://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm
/bin/yum install -y puppet-agent
/opt/puppetlabs/bin/puppet config set --section main server puppet.olindata.com
```
2. Configure/adjust Master
Ensure correct filename ymal file exists in bellow locations:
Ensure correct filename yaml file exists in below location:
```
/etc/puppetlabs/code/environments/production/hieradata/10.fqdn
```
```
/etc/puppetlabs/code/environments/production/hieradata/10.fqdn
```
3. Once this is done. Puppet agent.
3. Once this is done run puppet agent.
```
/opt/puppetlabs/bin/puppet agent -t
```
```
/opt/puppetlabs/bin/puppet agent -t
```
4. Content Setup
Currently gitlab / mattermost integration requires a bit of manual attention post-install including…
Creating opstheater-control repository
Pointing the puppetmaster’s code source to the opstheater-control repository on their gitlab installation
Creating demo repository from git@github.com:olindata/sample-ruby-project.git
Setup mattermost
Enable mattermost team creation
Log into mattermost with root user
Create a team
Make the team a public team
disable team creation
TEST IT :P
Create integration
Copy/paste integration URL into gitlab for build notifications
Currently gitlab / mattermost integration requires a bit of manual attention post-install including…
Creating opstheater-control repository
Pointing the puppetmaster’s code source to the opstheater-control repository on their gitlab installation
Creating demo repository from git@github.com:olindata/sample-ruby-project.git
Setup mattermost
Enable mattermost team creation
Log into mattermost with root user
Create a team
Make the team a public team
disable team creation
TEST IT :P
Create integration
Copy/paste integration URL into gitlab for build notifications
......@@ -2,30 +2,23 @@
1. Configure repo and install puppet.
```
/bin/yum install -y epel-release http://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm
/bin/yum install -y puppet-agent
/opt/puppetlabs/bin/puppet config set --section main server puppet.olindata.com
```
```
/bin/yum install -y epel-release http://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm
/bin/yum install -y puppet-agent
/opt/puppetlabs/bin/puppet config set --section main server puppet.olindata.com
```
2. Configure/adjust Master
* Update `/etc/puppetlabs/code/environment/production/hieradata/60-.opstheater.yaml` in icinga section to reflect correct name of icinga on the below line.
* Also update the ipaddress to the public ipaddress.
* Update `/etc/puppetlabs/code/environment/production/hieradata/50.common.yaml` in icinga section to reflect correct name of icinga on the below line.
* Also update the ipaddress to the public ipaddress.
```
'opstheater::icinga::fqdn': "icinga.%{hiera('opstheater::domain')}"
'opstheater::icinga::ipaddress': '37.139.31.34'
```
```
'opstheater::icinga::fqdn': "icinga.%{hiera('opstheater::domain')}"
'opstheater::icinga::ipaddress': '37.139.31.34'
```
* update site.pp to reflect correct name.
```
# runs standalone monitoring setup with icinga2 and icinga web2
node 'icinga.olindata.com' {
include opstheater::role::monitoring::standalone
}
```
* in the foreman, assign the role `opstheater::role::monitoring::standalone` to the node
3. Run puppet agent
......
......@@ -2,54 +2,54 @@
1. Configure repo and install puppet.
```
/bin/yum install -y epel-release http://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm
/bin/yum install -y puppet-agent
/opt/puppetlabs/bin/puppet config set --section main server puppet.olindata.com
```
```
/bin/yum install -y epel-release http://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm
/bin/yum install -y puppet-agent
/opt/puppetlabs/bin/puppet config set --section main server puppet.olindata.com
```
2. Configure/adjust Master
update /etc/puppetlabs/code/environment/production/hieradata/60-.opstheater.yaml to reflect myqsl server ip.
update /etc/puppetlabs/code/environment/production/hieradata/50.common.yaml to override parameters from 60.opstheater.yaml, like the myqsl server ip.
```
##########################
## MySQL related settings
##########################
```
##########################
## MySQL related settings
##########################
## MySQL related settings
# Variable: opstheater::mysql::fqdn
# Description:
# Default value: "mysql.%{hiera('opstheater::domain')}"
'opstheater::mysql::fqdn': "mysql.%{hiera('opstheater::domain')}"
## MySQL related settings
# Variable: opstheater::mysql::fqdn
# Description:
# Default value: "mysql.%{hiera('opstheater::domain')}"
'opstheater::mysql::fqdn': "mysql.%{hiera('opstheater::domain')}"
# Variable: opstheater::mysql::ipaddress
# Description:
# Default value: '10.20.1.60'
'opstheater::mysql::ipaddress': '10.129.2.113'
# Variable: opstheater::mysql::ipaddress
# Description:
# Default value: '10.20.1.60'
'opstheater::mysql::ipaddress': '10.129.2.113'
# Variable: opstheater::mysql::whitelist_range
# Description:
# Default value: '10.20.1.%'
'opstheater::mysql::whitelist_range': '10.129.%'
```
# Variable: opstheater::mysql::whitelist_range
# Description:
# Default value: '10.20.1.%'
'opstheater::mysql::whitelist_range': '10.129.%'
```
3. Run Puppet Agent
This will throw error for the 1st time as below but on the second run it works fine.
```
nfo: /Stage[main]/Mysql::Server::Config/File[mysql-config-file]: Filebucketed /etc/my.cnf to puppet with sum 80e1eb23d5fbd77fc0ff681b0f0df297
Notice: /Stage[main]/Mysql::Server::Config/File[mysql-config-file]/content: content changed '{md5}80e1eb23d5fbd77fc0ff681b0f0df297' to '{md5}3ab13aa54c001fe3966a08ae49c5517d'
Notice: Disabling SSL is evil! You should never ever do this except if you are forced to use a mysql version compiled without SSL support
Notice: /Stage[main]/Mysql::Server::Config/Notify[ssl-disable]/message: defined 'message' as 'Disabling SSL is evil! You should never ever do this except if you are forced to use a mysql version compiled without SSL support'
Error: Could not start Service[mysqld]: Execution of '/usr/bin/systemctl start mysqld' returned 1: Job for mysqld.service failed because the control process exited with error code. See "systemctl status mysqld.service" and "journalctl -xe" for details.
Error: /Stage[main]/Mysql::Server::Service/Service[mysqld]/ensure: change from stopped to running failed: Could not start Service[mysqld]: Execution of '/usr/bin/systemctl start mysqld' returned 1: Job for mysqld.service failed because the control process exited with error code. See "systemctl status mysqld.service" and "journalctl -xe" for details.
Notice: /Stage[main]/Opstheater::Profile::Mysql/File[/var/log/mysql]/ensure: created
Notice: /Stage[main]/Mysql::Server::Service/File[/var/log/mysql/error.log]/ensure: created
Info: Class[Mysql::Server::Service]: Unscheduling all events on Class[Mysql::Server::Service]
Notice: /Stage[main]/Mysql::Server::Root_password/Exec[remove install pass]: Dependency Service[mysqld] has failures: true
Warning: /Stage[main]/Mysql::Server::Root_password/Exec[remove install pass]: Skipping because of failed dependencies
Info: Creating state file /opt/puppetlabs/puppet/cache/state/state.yaml
Error: Failed to apply catalog: Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
```
This will throw error for the 1st time as below but on the second run it works fine.
```
Info: /Stage[main]/Mysql::Server::Config/File[mysql-config-file]: Filebucketed /etc/my.cnf to puppet with sum 80e1eb23d5fbd77fc0ff681b0f0df297
Notice: /Stage[main]/Mysql::Server::Config/File[mysql-config-file]/content: content changed '{md5}80e1eb23d5fbd77fc0ff681b0f0df297' to '{md5}3ab13aa54c001fe3966a08ae49c5517d'
Notice: Disabling SSL is evil! You should never ever do this except if you are forced to use a mysql version compiled without SSL support
Notice: /Stage[main]/Mysql::Server::Config/Notify[ssl-disable]/message: defined 'message' as 'Disabling SSL is evil! You should never ever do this except if you are forced to use a mysql version compiled without SSL support'
Error: Could not start Service[mysqld]: Execution of '/usr/bin/systemctl start mysqld' returned 1: Job for mysqld.service failed because the control process exited with error code. See "systemctl status mysqld.service" and "journalctl -xe" for details.
Error: /Stage[main]/Mysql::Server::Service/Service[mysqld]/ensure: change from stopped to running failed: Could not start Service[mysqld]: Execution of '/usr/bin/systemctl start mysqld' returned 1: Job for mysqld.service failed because the control process exited with error code. See "systemctl status mysqld.service" and "journalctl -xe" for details.
Notice: /Stage[main]/Opstheater::Profile::Mysql/File[/var/log/mysql]/ensure: created
Notice: /Stage[main]/Mysql::Server::Service/File[/var/log/mysql/error.log]/ensure: created
Info: Class[Mysql::Server::Service]: Unscheduling all events on Class[Mysql::Server::Service]
Notice: /Stage[main]/Mysql::Server::Root_password/Exec[remove install pass]: Dependency Service[mysqld] has failures: true
Warning: /Stage[main]/Mysql::Server::Root_password/Exec[remove install pass]: Skipping because of failed dependencies
Info: Creating state file /opt/puppetlabs/puppet/cache/state/state.yaml
Error: Failed to apply catalog: Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
```
......@@ -11,14 +11,13 @@
- [ ] Kibana
- [ ] Check if you are able to send message using Mattermost.
- [ ] Check if you are able to create a user in Foreman and can login with that user.
- [ ] Check if you can see the logs coming up for OpsTheater components in Icinga Web UI.
- [ ] Check if you can see the metrics coming up for OpsTheater components in Icinga Web UI.
- [ ] Try to search for a host in Kibana and check if you are getting logs of that host.
- [ ] Check if you can create users and usergroups in Icinga
- [ ] Login to Grafana web UI and check if you can see different metrics in Grafana
- [ ] Login to grafana and check that you can see two datasources. Edit each one of them and press "Test Connection" to make sure they work.
- [ ] Login to Kibana web UI and try to search for logs having source as "/var/log/puppetlabs/puppetserver/puppetserver-access.log". Check if the value of field timestamp is same as the time given in message field
**Release specific Checks :**
*These checks will change on every release and will be more specific for the new changes that we are introducing in this release.*
- [ ] Login to Grafana web UI and check if you can see different metrics in Grafana
- [ ] Login to grafana and check that you can see two datasources. Edit each one of them and press "Test Connection" to make sure they work.
- [ ] Login to Kibana web UI and try to search for logs having source as "/var/log/puppetlabs/puppetserver/puppetserver-access.log". Check if the value of field timestamp is same as the time given in message field
This diff is collapsed.
......@@ -4,4 +4,4 @@ Documentation for OpsTheater
## Getting started
If you wish to try out OpsTheater in a development setup, please look at the instructions here: https://gitlab.olindata.com/opstheater/opstheater-docs/blob/master/1.7/deployment/development.md
If you wish to try out OpsTheater in a development setup, please look at the instructions here: <https://gitlab.olindata.com/opstheater/opstheater-docs/blob/master/2.0/deployment/development.md>
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment