-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy paths3.txt
More file actions
196 lines (145 loc) · 9.2 KB
/
s3.txt
File metadata and controls
196 lines (145 loc) · 9.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
[1mLocalPath[22m: represents the path of a local file or directory. It can be
written as an absolute path or relative path.
[1mS3Uri[22m: represents the location of a S3 object, prefix, or bucket. This
must be written in the form [1ms3://mybucket/mykey [22mwhere [1mmybucket [22mis the
specified S3 bucket, [1mmykey [22mis the specified S3 key. The path argument
must begin with [1ms3:// [22min order to denote that the path argument refers
to a S3 object. Note that prefixes are separated by forward slashes.
For example, if the S3 object [1mmyobject [22mhad the prefix [1mmyprefix[22m, the S3
key would be [1mmyprefix/myobject[22m, and if the object was in the bucket
[1mmybucket[22m, the [1mS3Uri [22mwould be [1ms3://mybucket/myprefix/myobject[22m.
[1mS3Uri [22malso supports S3 access points. To specify an access point, this
value must be of the form [1ms3://<access-point-arn>/<key>[22m. For example if
the access point [1mmyaccesspoint [22mto be used has the ARN:
[1marn:aws:s3:us-west-2:123456789012:accesspoint/myaccesspoint [22mand the
object being accessed has the key [1mmykey[22m, then the [1mS3URI [22mused must be:
[1ms3://arn:aws:s3:us-west-2:123456789012:accesspoint/myaccesspoint/mykey[22m.
Similar to bucket names, you can also use prefixes with access point
ARNs for the [1mS3Uri[22m. For example:
[1ms3://arn:aws:s3:us-west-2:123456789012:accesspoint/myaccesspoint/mypre-[0m
[1mfix/[0m
The higher level [1ms3 [22mcommands do [1mnot [22msupport access point object ARNs.
For example, if the following was specified:
[1ms3://arn:aws:s3:us-west-2:123456789012:accesspoint/myaccess-[0m
[1mpoint/object/mykey [22mthe [1mS3URI [22mwill resolve to the object key
[1mobject/mykey[0m
[1mOrder of Path Arguments[0m
Every command takes one or two positional path arguments. The first
path argument represents the source, which is the local file/directory
or S3 object/prefix/bucket that is being referenced. If there is a
second path argument, it represents the destination, which is the local
file/directory or S3 object/prefix/bucket that is being operated on.
Commands with only one path argument do not have a destination because
the operation is being performed only on the source.
[1mSingle Local File and S3 Object Operations[0m
Some commands perform operations only on single files and S3 objects.
The following commands are single file/object operations if no [1m--recur-[0m
[1msive [22mflag is provided.
+o [1mcp[0m
+o [1mmv[0m
+o [1mrm[0m
For this type of operation, the first path argument, the source, must
exist and be a local file or S3 object. The second path argument, the
destination, can be the name of a local file, local directory, S3
object, S3 prefix, or S3 bucket.
The destination is indicated as a local directory, S3 prefix, or S3
bucket if it ends with a forward slash or back slash. The use of slash
depends on the path argument type. If the path argument is a [1mLocal-[0m
[1mPath[22m, the type of slash is the separator used by the operating system.
If the path is a [1mS3Uri[22m, the forward slash must always be used. If a
slash is at the end of the destination, the destination file or object
will adopt the name of the source file or object. Otherwise, if there
is no slash at the end, the file or object will be saved under the name
provided. See examples in [1mcp [22mand [1mmv [22mto illustrate this description.
[1mDirectory and S3 Prefix Operations[0m
Some commands only perform operations on the contents of a local direc-
tory or S3 prefix/bucket. Adding or omitting a forward slash or back
slash to the end of any path argument, depending on its type, does not
affect the results of the operation. The following commands will
always result in a directory or S3 prefix/bucket operation:
+o [1msync[0m
+o [1mmb[0m
+o [1mrb[0m
+o [1mls[0m
[1mUse of Exclude and Include Filters[0m
Currently, there is no support for the use of UNIX style wildcards in a
command's path arguments. However, most commands have [1m--exclude[0m
[1m"<value>" [22mand [1m--include "<value>" [22mparameters that can achieve the
desired result. These parameters perform pattern matching to either
exclude or include a particular file or object. The following pattern
symbols are supported.
+o [1m*[22m: Matches everything
+o [1m?[22m: Matches any single character
+o [1m[sequence][22m: Matches any character in [1msequence[0m
+o [1m[!sequence][22m: Matches any character not in [1msequence[0m
Any number of these parameters can be passed to a command. You can do
this by providing an [1m--exclude [22mor [1m--include [22margument multiple times,
e.g. [1m--include "*.txt" --include "*.png"[22m. When there are multiple
filters, the rule is the filters that appear later in the command take
precedence over filters that appear earlier in the command. For exam-
ple, if the filter parameters passed to the command were
--exclude "*" --include "*.txt"
All files will be excluded from the command except for files ending
with [1m.txt [22mHowever, if the order of the filter parameters was changed
to
--include "*.txt" --exclude "*"
All files will be excluded from the command.
Each filter is evaluated against the [1msource directory[22m. If the source
location is a file instead of a directory, the directory containing the
file is used as the source directory. For example, suppose you had the
following directory structure:
/tmp/foo/
.git/
|---config
|---description
foo.txt
bar.txt
baz.jpg
In the command [1maws s3 sync /tmp/foo s3://bucket/ [22mthe source directory
is [1m/tmp/foo[22m. Any include/exclude filters will be evaluated with the
source directory prepended. Below are several examples to demonstrate
this.
Given the directory structure above and the command [1maws s3 cp /tmp/foo[0m
[1ms3://bucket/ --recursive --exclude ".git/*"[22m, the files [1m.git/config [22mand
[1m.git/description [22mwill be excluded from the files to upload because the
exclude filter [1m.git/* [22mwill have the source prepended to the filter.
This means that:
/tmp/foo/.git/* -> /tmp/foo/.git/config (matches, should exclude)
/tmp/foo/.git/* -> /tmp/foo/.git/description (matches, should exclude)
/tmp/foo/.git/* -> /tmp/foo/foo.txt (does not match, should include)
/tmp/foo/.git/* -> /tmp/foo/bar.txt (does not match, should include)
/tmp/foo/.git/* -> /tmp/foo/baz.jpg (does not match, should include)
The command [1maws s3 cp /tmp/foo/ s3://bucket/ --recursive --exclude[0m
[1m"ba*" [22mwill exclude [1m/tmp/foo/bar.txt [22mand [1m/tmp/foo/baz.jpg[22m:
/tmp/foo/ba* -> /tmp/foo/.git/config (does not match, should include)
/tmp/foo/ba* -> /tmp/foo/.git/description (does not match, should include)
/tmp/foo/ba* -> /tmp/foo/foo.txt (does not match, should include)
/tmp/foo/ba* -> /tmp/foo/bar.txt (matches, should exclude)
/tmp/foo/ba* -> /tmp/foo/baz.jpg (matches, should exclude)
Note that, by default, [4mall[24m [4mfiles[24m [4mare[24m [4mincluded[24m. This means that provid-
ing [1monly [22man [1m--include [22mfilter will not change what files are trans-
ferred. [1m--include [22mwill only re-include files that have been excluded
from an [1m--exclude [22mfilter. If you only want to upload files with a par-
ticular extension, you need to first exclude all files, then re-include
the files with the particular extension. This command will upload [1monly[0m
files ending with [1m.jpg[22m:
aws s3 cp /tmp/foo/ s3://bucket/ --recursive --exclude "*" --include "*.jpg"
If you wanted to include both [1m.jpg [22mfiles as well as [1m.txt [22mfiles you can
run:
aws s3 cp /tmp/foo/ s3://bucket/ --recursive \
--exclude "*" --include "*.jpg" --include "*.txt"
[1mSYNOPSIS[0m
aws s3 <Command> [<Arg> ...]
[1mOPTIONS[0m
[4mNone[0m
[1mAVAILABLE COMMANDS[0m
+o cp
+o ls
+o mb
+o mv
+o presign
+o rb
+o rm
+o sync
+o website
S3()