README.md 11.5 KB
Newer Older
Daisuke Fujita's avatar
Daisuke Fujita committed
1
# Terraforming
Daisuke Fujita's avatar
Daisuke Fujita committed
2

Daisuke Fujita's avatar
Daisuke Fujita committed
3 4
[![Build Status](https://travis-ci.org/dtan4/terraforming.svg?branch=master)](https://travis-ci.org/dtan4/terraforming)
[![Code Climate](https://codeclimate.com/github/dtan4/terraforming/badges/gpa.svg)](https://codeclimate.com/github/dtan4/terraforming)
5
[![Coverage Status](https://coveralls.io/repos/github/dtan4/terraforming/badge.svg?branch=increase-test-cov-160528)](https://coveralls.io/github/dtan4/terraforming)
Daisuke Fujita's avatar
Daisuke Fujita committed
6
[![Dependency Status](https://gemnasium.com/dtan4/terraforming.svg)](https://gemnasium.com/dtan4/terraforming)
Daisuke Fujita's avatar
Daisuke Fujita committed
7
[![Gem Version](https://badge.fury.io/rb/terraforming.svg)](http://badge.fury.io/rb/terraforming)
8
[![MIT License](http://img.shields.io/badge/license-MIT-blue.svg?style=flat)](LICENSE)
9
[![Docker Repository on Quay.io](https://quay.io/repository/dtan4/terraforming/status "Docker Repository on Quay.io")](https://quay.io/repository/dtan4/terraforming)
Daisuke Fujita's avatar
Daisuke Fujita committed
10
[![Join the chat at https://gitter.im/dtan4/terraforming](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/dtan4/terraforming?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
Daisuke Fujita's avatar
Daisuke Fujita committed
11

Daisuke Fujita's avatar
Daisuke Fujita committed
12
Export existing AWS resources to [Terraform](https://terraform.io/) style (tf, tfstate)
Daisuke Fujita's avatar
Daisuke Fujita committed
13

Daisuke Fujita's avatar
Daisuke Fujita committed
14
- [Supported version](#supported-version)
Daisuke Fujita's avatar
Daisuke Fujita committed
15 16 17 18 19
- [Installation](#installation)
- [Prerequisites](#prerequisites)
- [Usage](#usage)
  -  [Export tf](#export-tf)
  -  [Export tfstate](#export-tfstate)
20
    -  [Example: Export all](#example-export-all)
Daisuke Fujita's avatar
Daisuke Fujita committed
21 22 23 24 25
- [Run as Docker container](#run-as-docker-container-)
- [Development](#development)
- [Contributing](#contributing)
- [License](#license)

Daisuke Fujita's avatar
Daisuke Fujita committed
26 27
## Supported version

Daisuke Fujita's avatar
Daisuke Fujita committed
28
Ruby 2.1 or higher
Daisuke Fujita's avatar
Daisuke Fujita committed
29

Daisuke Fujita's avatar
Daisuke Fujita committed
30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
## Installation

Add this line to your application's Gemfile:

```ruby
gem 'terraforming'
```

And then execute:

    $ bundle

Or install it yourself as:

    $ gem install terraforming

Daisuke Fujita's avatar
Daisuke Fujita committed
46 47
## Prerequisites

Daisuke Fujita's avatar
Daisuke Fujita committed
48 49 50
You need to set AWS credentials.

```bash
Daisuke Fujita's avatar
Daisuke Fujita committed
51 52
export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXX
export AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
53
export AWS_REGION=xx-yyyy-0
Daisuke Fujita's avatar
Daisuke Fujita committed
54
```
Daisuke Fujita's avatar
Daisuke Fujita committed
55

56 57 58 59 60 61 62 63 64 65 66 67
You can also specify credential profile in `~/.aws/credentials` by `--profile` option.

```bash
$ cat ~/.aws/credentials
[hoge]
aws_access_key_id = Hoge
aws_secret_access_key = FugaFuga

# Pass profile name by --profile option
$ terraforming s3 --profile hoge
```

Matt Gartman's avatar
Matt Gartman committed
68
You can force the AWS SDK to utilize the CA certificate that is bundled with the SDK for systems where the default OpenSSL certificate is not installed (e.g. Windows) by utilizing the `--use-bundled-cert` option.
69 70

```bash
71
PS C:\> terraforming ec2 --use-bundled-cert
72 73
```

Daisuke Fujita's avatar
Daisuke Fujita committed
74 75
## Usage

Daisuke Fujita's avatar
Daisuke Fujita committed
76
```bash
Daisuke Fujita's avatar
Daisuke Fujita committed
77 78
$ terraforming
Commands:
Daisuke Fujita's avatar
Daisuke Fujita committed
79
  terraforming alb             # ALB
80
  terraforming asg             # AutoScaling Group
Jacopo Scrinzi's avatar
Jacopo Scrinzi committed
81
  terraforming cwa             # CloudWatch Alarm
Daisuke Fujita's avatar
Daisuke Fujita committed
82 83 84
  terraforming dbpg            # Database Parameter Group
  terraforming dbsg            # Database Security Group
  terraforming dbsn            # Database Subnet Group
Daisuke Fujita's avatar
Daisuke Fujita committed
85
  terraforming ec2             # EC2
Daisuke Fujita's avatar
Daisuke Fujita committed
86 87
  terraforming ecc             # ElastiCache Cluster
  terraforming ecsn            # ElastiCache Subnet Group
88
  terraforming efs             # EFS File System
Stephen Muth's avatar
Stephen Muth committed
89
  terraforming eip             # EIP
Daisuke Fujita's avatar
Daisuke Fujita committed
90
  terraforming elb             # ELB
Daisuke Fujita's avatar
Daisuke Fujita committed
91
  terraforming help [COMMAND]  # Describe available commands or one specific command
Daisuke Fujita's avatar
Daisuke Fujita committed
92
  terraforming iamg            # IAM Group
Daisuke Fujita's avatar
Daisuke Fujita committed
93
  terraforming iamgm           # IAM Group Membership
Daisuke Fujita's avatar
Daisuke Fujita committed
94
  terraforming iamgp           # IAM Group Policy
95
  terraforming iamip           # IAM Instance Profile
Daisuke Fujita's avatar
Daisuke Fujita committed
96
  terraforming iamp            # IAM Policy
Daisuke Fujita's avatar
Daisuke Fujita committed
97
  terraforming iampa           # IAM Policy Attachment
Daisuke Fujita's avatar
Daisuke Fujita committed
98
  terraforming iamr            # IAM Role
99
  terraforming iamrp           # IAM Role Policy
Daisuke Fujita's avatar
Daisuke Fujita committed
100 101
  terraforming iamu            # IAM User
  terraforming iamup           # IAM User Policy
Manabu Sakai's avatar
Manabu Sakai committed
102
  terraforming igw             # Internet Gateway
Daisuke Fujita's avatar
Daisuke Fujita committed
103 104
  terraforming kmsa            # KMS Key Alias
  terraforming kmsk            # KMS Key
Daisuke Fujita's avatar
Daisuke Fujita committed
105
  terraforming lc              # Launch Configuration
Daisuke Fujita's avatar
Daisuke Fujita committed
106
  terraforming nacl            # Network ACL
107
  terraforming nat             # NAT Gateway
Stephen Muth's avatar
Stephen Muth committed
108
  terraforming nif             # Network Interface
Daisuke Fujita's avatar
Daisuke Fujita committed
109 110
  terraforming r53r            # Route53 Record
  terraforming r53z            # Route53 Hosted Zone
Daisuke Fujita's avatar
Daisuke Fujita committed
111
  terraforming rds             # RDS
Manabu Sakai's avatar
Manabu Sakai committed
112
  terraforming rs              # Redshift
k1LoW's avatar
k1LoW committed
113
  terraforming rt              # Route Table
k1LoW's avatar
k1LoW committed
114
  terraforming rta             # Route Table Association
Daisuke Fujita's avatar
Daisuke Fujita committed
115
  terraforming s3              # S3
116
  terraforming sg              # Security Group
Daisuke Fujita's avatar
Daisuke Fujita committed
117
  terraforming sn              # Subnet
milesrl's avatar
milesrl committed
118
  terraforming snst            # SNS Topic
milesrl's avatar
milesrl committed
119
  terraforming snss            # SNS Subscription
Manabu Sakai's avatar
Add SQS  
Manabu Sakai committed
120
  terraforming sqs             # SQS
Daisuke Fujita's avatar
Daisuke Fujita committed
121
  terraforming vgw             # VPN Gateway
Daisuke Fujita's avatar
Daisuke Fujita committed
122
  terraforming vpc             # VPC
Daisuke Fujita's avatar
Daisuke Fujita committed
123 124

Options:
Daisuke Fujita's avatar
Daisuke Fujita committed
125 126 127 128 129 130
  [--merge=MERGE]                                # tfstate file to merge
  [--overwrite], [--no-overwrite]                # Overwrite existng tfstate
  [--tfstate], [--no-tfstate]                    # Generate tfstate
  [--profile=PROFILE]                            # AWS credentials profile
  [--region=REGION]                              # AWS region
  [--use-bundled-cert], [--no-use-bundled-cert]  # Use the bundled CA certificate from AWS SDK
Daisuke Fujita's avatar
Daisuke Fujita committed
131
```
Daisuke Fujita's avatar
Daisuke Fujita committed
132

Daisuke Fujita's avatar
Daisuke Fujita committed
133 134
### Export tf

Daisuke Fujita's avatar
Daisuke Fujita committed
135
```bash
136
$ terraforming <resource> [--profile PROFILE]
Daisuke Fujita's avatar
Daisuke Fujita committed
137 138
```

Daisuke Fujita's avatar
Daisuke Fujita committed
139
(e.g. S3 buckets):
Daisuke Fujita's avatar
Daisuke Fujita committed
140 141 142 143 144

```bash
$ terraforming s3
```

145
```hcl
Daisuke Fujita's avatar
Daisuke Fujita committed
146 147 148 149 150 151 152 153 154 155 156
resource "aws_s3_bucket" "hoge" {
    bucket = "hoge"
    acl    = "private"
}

resource "aws_s3_bucket" "fuga" {
    bucket = "fuga"
    acl    = "private"
}
```

Daisuke Fujita's avatar
Daisuke Fujita committed
157 158
### Export tfstate

Daisuke Fujita's avatar
Daisuke Fujita committed
159
```bash
160
$ terraforming <resource> --tfstate [--merge TFSTATE_PATH] [--overwrite] [--profile PROFILE]
Daisuke Fujita's avatar
Daisuke Fujita committed
161 162 163
```

(e.g. S3 buckets):
Daisuke Fujita's avatar
Daisuke Fujita committed
164 165 166 167 168

```bash
$ terraforming s3 --tfstate
```

169
```json
Daisuke Fujita's avatar
Daisuke Fujita committed
170
{
Daisuke Fujita's avatar
Daisuke Fujita committed
171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200
  "version": 1,
  "serial": 1,
  "modules": {
    "path": [
      "root"
    ],
    "outputs": {
    },
    "resources": {
      "aws_s3_bucket.hoge": {
        "type": "aws_s3_bucket",
        "primary": {
          "id": "hoge",
          "attributes": {
            "acl": "private",
            "bucket": "hoge",
            "id": "hoge"
          }
        }
      },
      "aws_s3_bucket.fuga": {
        "type": "aws_s3_bucket",
        "primary": {
          "id": "fuga",
          "attributes": {
            "acl": "private",
            "bucket": "fuga",
            "id": "fuga"
          }
        }
Daisuke Fujita's avatar
Daisuke Fujita committed
201 202 203 204 205 206
      }
    }
  }
}
```

207
If you want to merge exported tfstate to existing `terraform.tfstate`, specify `--tfstate --merge=/path/to/terraform.tfstate` option.
208
You can overwrite existing `terraform.tfstate` by specifying `--overwrite` option together.
Daisuke Fujita's avatar
Daisuke Fujita committed
209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330

Existing `terraform.tfstate`:

```bash
# /path/to/terraform.tfstate

{
  "version": 1,
  "serial": 88,
  "remote": {
    "type": "s3",
    "config": {
      "bucket": "terraforming-tfstate",
      "key": "tf"
    }
  },
  "modules": {
    "path": [
      "root"
    ],
    "outputs": {
    },
    "resources": {
      "aws_elb.hogehoge": {
        "type": "aws_elb",
        "primary": {
          "id": "hogehoge",
          "attributes": {
            "availability_zones.#": "2",
            "connection_draining": "true",
            "connection_draining_timeout": "300",
            "cross_zone_load_balancing": "true",
            "dns_name": "hoge-12345678.ap-northeast-1.elb.amazonaws.com",
            "health_check.#": "1",
            "id": "hogehoge",
            "idle_timeout": "60",
            "instances.#": "1",
            "listener.#": "1",
            "name": "hoge",
            "security_groups.#": "2",
            "source_security_group": "default",
            "subnets.#": "2"
          }
        }
      }
    }
  }
}
```

To generate merged tfstate:

```bash
$ terraforming s3 --tfstate --merge=/path/to/tfstate
```

```json
{
  "version": 1,
  "serial": 89,
  "remote": {
    "type": "s3",
    "config": {
      "bucket": "terraforming-tfstate",
      "key": "tf"
    }
  },
  "modules": {
    "path": [
      "root"
    ],
    "outputs": {
    },
    "resources": {
      "aws_elb.hogehoge": {
        "type": "aws_elb",
        "primary": {
          "id": "hogehoge",
          "attributes": {
            "availability_zones.#": "2",
            "connection_draining": "true",
            "connection_draining_timeout": "300",
            "cross_zone_load_balancing": "true",
            "dns_name": "hoge-12345678.ap-northeast-1.elb.amazonaws.com",
            "health_check.#": "1",
            "id": "hogehoge",
            "idle_timeout": "60",
            "instances.#": "1",
            "listener.#": "1",
            "name": "hoge",
            "security_groups.#": "2",
            "source_security_group": "default",
            "subnets.#": "2"
          }
        }
      },
      "aws_s3_bucket.hoge": {
        "type": "aws_s3_bucket",
        "primary": {
          "id": "hoge",
          "attributes": {
            "acl": "private",
            "bucket": "hoge",
            "id": "hoge"
          }
        }
      },
      "aws_s3_bucket.fuga": {
        "type": "aws_s3_bucket",
        "primary": {
          "id": "fuga",
          "attributes": {
            "acl": "private",
            "bucket": "fuga",
            "id": "fuga"
          }
        }
      }
    }
  }
}
```
Daisuke Fujita's avatar
Daisuke Fujita committed
331

Daisuke Fujita's avatar
Daisuke Fujita committed
332 333 334 335 336 337 338 339 340 341 342
After writing exported tf and tfstate to files, execute `terraform plan` and check the result.
There should be no diff.

```bash
$ terraform plan
No changes. Infrastructure is up-to-date. This means that Terraform
could not detect any differences between your configuration and
the real physical resources that exist. As a result, Terraform
doesn't need to do anything.
```

343 344 345 346 347
#### Example: Export all
Example assuming you want to export everything from us-west-2 and you are using ~/.aws/credentials with a `default` profile
```bash
export AWS_REGION=us-west-2
terraforming help | grep terraforming | grep -v help | awk '{print "terraforming", $2, "--profile", "default", ">", $2".tf";}' | bash
348 349
# find files that only have 1 empty line (likely nothing in AWS)
find . -type f -name '*.tf' | xargs wc -l | grep ' 1 .'
350 351
```

Daisuke Fujita's avatar
Daisuke Fujita committed
352
## Run as Docker container [![Docker Repository on Quay.io](https://quay.io/repository/dtan4/terraforming/status "Docker Repository on Quay.io")](https://quay.io/repository/dtan4/terraforming)
353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369

Terraforming Docker Image is available at [quay.io/dtan4/terraforming](https://quay.io/repository/dtan4/terraforming) and developed at [dtan4/dockerfile-terraforming](https://github.com/dtan4/dockerfile-terraforming).

Pull the Docker image:

```bash
$ docker pull quay.io/dtan4/terraforming:latest
```

And then run Terraforming as a Docker container:

```bash
$ docker run \
    --rm \
    --name terraforming \
    -e AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXX \
    -e AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
370
    -e AWS_REGION=xx-yyyy-0 \
371 372 373 374
    quay.io/dtan4/terraforming:latest \
    terraforming s3
```

Daisuke Fujita's avatar
Daisuke Fujita committed
375 376
## Development

Daisuke Fujita's avatar
Daisuke Fujita committed
377
After checking out the repo, run `script/setup` to install dependencies. Then, run `script/console` for an interactive prompt that will allow you to experiment.
Daisuke Fujita's avatar
Daisuke Fujita committed
378 379 380 381 382

To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release` to create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).

## Contributing

383 384
Please read [Contribution Guide](CONTRIBUTING.md) at first.

Daisuke Fujita's avatar
Daisuke Fujita committed
385 386 387 388 389 390
1. Fork it ( https://github.com/dtan4/terraforming/fork )
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create a new Pull Request

Daisuke Fujita's avatar
Daisuke Fujita committed
391
## License
Daisuke Fujita's avatar
Daisuke Fujita committed
392 393

[![MIT License](http://img.shields.io/badge/license-MIT-blue.svg?style=flat)](LICENSE)