~bzr-pqm/bzr/bzr.dev

4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
1
=========================
2
Bazaar Windows EC2 Server
3
=========================
4
5
We have an Amazon EC2 virtual machine called Desolation_ for
6
building Windows packages and general testing on Windows.  As of
7
2009-02-19, this is just experimental and this is a draft specification,
8
but we aim to use it for the production Windows installer build of 1.13 in
9
March.
10
4853.1.1 by Patrick Regan
Removed trailing whitespace from files in doc directory
11
See also:
4012.2.2 by Martin Pool
ReST syntax fix
12
13
* `Bazaar Developer Documentation Catalog <index.html>`_.
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
14
15
16
.. _Desolation: http://en.wikipedia.org/wiki/Desolation_Island
17
18
19
Goals
20
=====
21
22
* The instance is only running (and incurring charges) when it's needed
4853.1.1 by Patrick Regan
Removed trailing whitespace from files in doc directory
23
  for testing or packaging.
24
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
25
* It can be started or stopped by anyone on the team using a
26
  straightforward script.
27
28
* Multiple people can get into the same instance at the same time, e.g.
29
  if one person needs to pass work on to some one else.
30
31
* We keep snapshot of the OS and tool chain so that we can roll back if
32
  we need to.
33
34
* bzr branches and similar information are kept on stable storage that
35
  survives rollbacks of the OS state, and that can be backed up.
36
37
Later on we may try automated Windows testing in a similar setup.
38
39
40
Approach
41
========
42
43
The working disk and the AMI images are stored in one person's account for
4853.1.1 by Patrick Regan
Removed trailing whitespace from files in doc directory
44
billing purposes.
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
45
46
Ideally we want to give other people access to run this machine without
47
giving full access to the account.  I'm not sure if that's feasible.  If
48
it's not, we might need to allow people to launch the image within their
49
own account; this may be problematic if the shared volume is already in
50
use by someone else.
51
52
I don't think it's possible to have an EBS that's shared across accounts,
53
and they can't be attached to multiple running instances.  So for now it's
54
probably best to just ignore the concept and store the working data on the
55
instance's local storage, and to copy things up e.g. to Launchpad as
56
required.
57
58
On this machine, ``C:`` should be used only for the Windows system files,
59
``D:`` for installed programs and working directories, and other drive
60
letters can be used later for mounting EBS storage if desired.
61
62
Through ``ec2-modify-image-attribute`` we can allow nominated users to
63
access an existing image.  We need to have their AWS opaque ID.
64
65
Through ``ec2-bundle-image`` we can make a new snapshot at any point,
66
which will be stored into the current user's S3 account.
67
68
We'll (probably) have one shared account for running builds which is also
69
an administrator for ease of installing software.
70
71
You do need to have an RSA keypair to get the initial password for a
4165.3.1 by Robert Collins
Various tweaks to ec2 stuff, and make it less windows only.
72
Windows machine, even though you can't use it to log in later.
4165.3.2 by Robert Collins
Review feedback.
73
``ec2-get-password`` takes the full path to the private key to obtain the
74
password from Amazon, and ``ec2-add-keypair`` creates a named keypair at
4165.3.1 by Robert Collins
Various tweaks to ec2 stuff, and make it less windows only.
75
Amazon and returns the private path. One keypair is all that is needed.
76
This is distinct from the account identifier - likely due to the different
5538.2.2 by Zearin
Continued capitalization fixes ([S]FTP, SSH).
77
toolchains in use (the keypairs are used for unix SSH keys, and I (Robert)
4165.3.1 by Robert Collins
Various tweaks to ec2 stuff, and make it less windows only.
78
suspect a rather unix friendly core at Amazon).
4853.1.1 by Patrick Regan
Removed trailing whitespace from files in doc directory
79
Once a custom image is made with a saved password, you can skip using
4165.3.2 by Robert Collins
Review feedback.
80
``ec2-get-password`` (which is only needed for Windows anyway).
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
81
82
It would be nice if rdesktop could use private key authentication but
83
apparently not.
84
85
Should check how the Launchpad ec2test scripts work.
86
87
88
89
Procedures
90
==========
91
92
Preparation
93
-----------
94
95
* Be in the bzr core team.  If you are interested in helping with
96
  Windows packaging, testing or development just ask.
97
98
* Install the
99
  `Amazon EC2 API tools`_ (needs-packaging `bug 330930`_)
100
4165.3.1 by Robert Collins
Various tweaks to ec2 stuff, and make it less windows only.
101
* Create an Amazon Web Services account, sign up for S3 and EC2, and do
102
  the various steps to create authentication devices.
103
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
104
* Create a private key and certificate for yourself.
105
  Check these environment variables are set and exported, e.g. by setting
106
  them in the file ``~/.aws``.  Make sure the files are private.::
107
4165.3.1 by Robert Collins
Various tweaks to ec2 stuff, and make it less windows only.
108
    export EC2_PRIVATE_KEY=~/.ec2/pk-XXXXXX.pem
109
    export EC2_CERT=~/.ec2/cert-XXXXXX.pem
110
    export EC2_HOME=~/build/ec2-api-tools-1.3-30349
111
    export AWS_SECRET_ACCESS_KEY=XXXXXXXXX
112
    export AWS_ACCESS_KEY_ID=XXXXXXXXXXX
113
    export EC2_KEYPAIR_NAME=XXXXXXXXX
114
    export PATH=$PATH:$EC2_HOME/bin
115
    export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
116
    ssh-add ~/.ec2/id_rsa
117
118
  You can now '. ~/.aws' to get the ec2 commands available.
119
120
* (Unix images only) run ec2-add-keypair SOMENAME, e.g. 'bzr'. Put the
121
  result (minus the first line) somewhere like ~/.ec2/id_rsa and chmod go-rw.
122
123
* A useful Unix image is `ami-bdfe19d4`_, Eric Hammonds 64-bit Ubuntu image.
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
124
125
* Install the rdesktop client, to actually access the machine.
126
127
* Possibly read some of the `EC2 documentation`_ for background.
128
   
4634.166.3 by Vincent Ladeuil
The last remaining urls mentioning edge.
129
.. _`bug 330930`: https://bugs.launchpad.net/ubuntu/+bug/330930 
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
130
.. _`Amazon EC2 API tools`: 
131
     http://developer.amazonwebservices.com/connect/entry.jspa?externalID=368&categoryID=88
132
.. _`EC2 documentation`: http://aws.amazon.com/
4165.3.1 by Robert Collins
Various tweaks to ec2 stuff, and make it less windows only.
133
.. _`ami-bdfe19d4`:
134
   http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1762&categoryID=101
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
135
136
* Create a security group for your that allows rdesktop access and icmp with::
137
138
    ec2-add-group desolation-group -d 'bzr win32 build machine'
139
    ec2-authorize desolation-group -p 3389 -s 1.2.3.4/32
140
    ec2-authorize desolation-group -t -1:-1 -P icmp
4853.1.1 by Patrick Regan
Removed trailing whitespace from files in doc directory
141
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
142
  Add your public IP there.  You can repeat that command to allow others
143
  in.
144
145
146
To start up an instance
147
-----------------------
148
149
1. Get the right AMI image ID from another developer.
150
151
1. Start the instance::
4853.1.1 by Patrick Regan
Removed trailing whitespace from files in doc directory
152
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
153
     ec2-run-instances $image_id -g desolation-group
154
155
   This will print out some information including the image id, something
4853.1.1 by Patrick Regan
Removed trailing whitespace from files in doc directory
156
   like ``i-31a74258``.
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
157
158
1. Actually starting the machine will take a few minutes.  Once it's in
159
   the *running* state, get the machine's public IP with ::
160
161
     ec2-describe-instances
162
163
1. and then connect ::
164
165
     rdesktop -g 1200x850 -u Administrator $machine_ip
166
167
Don't forget to shut it down when you're done, and check with
168
``ec2-describe-instances`` that it did terminate.
169
170
171
To save a system snapshot as an image
172
-------------------------------------
173
4853.1.1 by Patrick Regan
Removed trailing whitespace from files in doc directory
174
1. Bundle the current state.  *Doing this will reboot the machine.*
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
175
   You need to choose a unique s3 bucket name,
4853.1.1 by Patrick Regan
Removed trailing whitespace from files in doc directory
176
   typically based on a domain or email address, which can contain
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
177
   any number of images.  You also need a name unique within the bucket
4853.1.1 by Patrick Regan
Removed trailing whitespace from files in doc directory
178
   for this image, like ``desolation-vs2008-20090219``.  And finally
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
179
   it needs your AWS S3 access key and secret key, which should be set in
180
   ``~/.aws``::
4853.1.1 by Patrick Regan
Removed trailing whitespace from files in doc directory
181
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
182
      ec2-bundle-instance -b ec2.sourcefrog.net \
183
          -p desolation-vs2008-2009021 \
184
          -o "$AWS_ACCESS_KEY_ID" \
4853.1.1 by Patrick Regan
Removed trailing whitespace from files in doc directory
185
          -w "$AWS_SECRET_ACCESS_KEY"
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
186
187
1.  This will take several minutes:  You can check progress with ::
188
189
      ec2-describe-bundle-tasks
190
191
1. Register the files as an image, e.g.::
192
193
      ec2-register ec2.sourcefrog.net/desolation-vs2008-2009021
194
195
    This will give you an AMI id for the image.
196
197
1. Give access to other team members identified by their Amazon account id::
198
199
      ec2-modify-image-attributes $ami_id -l -a 123412341234
200
201
4165.3.1 by Robert Collins
Various tweaks to ec2 stuff, and make it less windows only.
202
Management console (useful!)
203
----------------------------
204
205
https://console.aws.amazon.com/ec2/home
206
4012.2.1 by Martin Pool
Documentation on using EC2 Windows VM
207
..
208
   vim: ft=rst tw=74 ai