[
  {
    "path": "CONTRIBUTORS",
    "content": "Moriyoshi Koizumi\nDmitry Chepurovskiy\n"
  },
  {
    "path": "LICENSE",
    "content": "Copyright (c) 2018 Moriyoshi Koizumi\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of\nthis software and associated documentation files (the \"Software\"), to deal in\nthe Software without restriction, including without limitation the rights to\nuse, copy, modify, merge, publish, distribute, sublicense, and/or sell copies\nof the Software, and to permit persons to whom the Software is furnished to do\nso, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n\n"
  },
  {
    "path": "README.md",
    "content": "# s3-sftp-proxy\n\n`s3-sftp-proxy` is a tiny program that exposes the resources on your AWS S3 buckets through SFTP protocol.\n\n## Usage\n\n```\nUsage of s3-sftp-proxy:\n  -bind string\n        listen on addr:port\n  -config string\n        configuration file (default \"s3-sftp-proxy.toml\")\n  -debug\n        turn on debugging output\n```\n\n* `-bind`\n\n\tSpecifies the local address and port to listen on.  This overrides the value of `bind` in the configuration file.  If it is not present in the configuration file either, it defaults to `:10022`.\n\n* `-config`\n\n\tSpecifies the path to the configuration file.  It defaults to \"./s3-sftp-config.toml\" if not given.\n\n* `-debug`\n\n\tTurn on debug logging.  The output will be more verbose.\n\n \n## Configuation\n\nThe configuration file is in [TOML](https://github.com/toml-lang/toml) format.  Refer to that page for the detailed explanation of the syntax.\n\n### Top level\n\n```toml\nhost_key_file = \"./host_key\"\nbind = \"localhost:10022\"\nbanner = \"\"\"\nWelcome to my SFTP server\n\"\"\"\nreader_lookback_buffer_size = 1048576\nreader_min_chunk_size = 262144\nlister_lookback_buffer_size = 100\n\n# buckets and authantication settings follow...\n```\n\n* `host_key_file` (required)\n\n\tSpecifies the path to the host key file (private key).\n\n\tThe host key can be generated with `ssh-keygen` command:\n\n\t```sh\n\tssh-keygen -f host_key\n\t```\n\n* `bind` (optional, defaults to `\":10022\"`)\n\n\tSpecifies the local address and port to listen on.\n\n* `banner` (optional, defaults to an empty string)\n\n\tA banner is a message text that will be sent to the client when the connection is esablished to the server prior to any authentication steps.\n\n* `reader_lookback_buffer_size` (optional, defaults to `1048576`)\n\n\tSpecifies the size of the buffer used to keep several amounts of data read from S3 for later access to it.  The reason why such buffer is necessary is that SFTP protocol requires the data should be sent or retrieved on a random-access basis (i.e. each request contains an offset) while those coming from S3 is actually fetched in a streaming manner.   In that we have to emulate block storage access for S3 objects, but chances are we don't need to hold the entire data with the reasonable SFTP clients.\n\n* `reader_min_chunk_size` (optional, defaults to `262144`)\n\n\tSpecifies the amount of data fetched from S3 at once.  Increase the value when you experience quite a poor performance.\n\n* `lister_lookback_buffer_size` (optional, defalts to `100`)\n\n\tContrary to the people's expectation, SFTP also requires file listings to be retrieved in random-access as well.\n\n* `buckets` (required)\n\n\t`buckets` contains records for bucket declarations.  See [Bucket Settings](#bucket-settings) for detail.\n\n* `auth`\n\n\t`auth` contains records for authenticator configurations.  See [Authenticator Settings](#authenticator-settings) for detail.\n\n### Bucket Settings\n\n```toml\n[buckets.test]\nendpoint = \"http://endpoint\"\ns3_force_path_style = true\ndisable_ssl = false\nbucket = \"BUCKET\"\nkey_prefix = \"PREFIX\"\nbucket_url = \"s3://BUCKET/PREFIX\"\nprofile = \"profile\"\nregion = \"ap-northeast-1\"\nmax_object_size = 65536\nwritable = false\nreadable = true\nlistable = true\nauth = \"test\"\nserver_side_encryption = \"kms\"\nsse_customer_key = \"\"\nsse_kms_key_id = \"\"\nkeyboard_interactive_auth = false\n\n[buckets.test.credentials]\naws_access_key_id = \"aaa\"\naws_secret_access_key = \"bbb\"\n```\n\n* `endpoint` (optional)\n\tSpecifies s3 endpoint (server) different from AWS.\n\n* `s3_force_path_style` (optional)\n    This option should be set to `true` if ypu use endpount different from AWS.\n    \n\tSet this to `true` to force the request to use path-style addressing, i.e., `http://s3.amazonaws.com/BUCKET/KEY`. By default, the S3 client will use virtual hosted bucket addressing when possible (`http://BUCKET.s3.amazonaws.com/KEY`).\n\n* `disable_ssl` (optional)\n\tSet this to `true` to disable SSL when sending requests.\n\n* `bucket` (required when `bucket_url` is unspecified)\n\n\tSpecifies the bucket name.\n\n* `key_prefix` (required when `bucket_url` is unspecified)\n\t\n\tSpecifies the prefix prepended to the file path sent from the client.  The key string is derived as follows:\n\n\t\t`key` = `key_prefix` + `path`\n\n* `bucket_url` (required when `bucket` is unspecified)\n\n\tSpecifies both the bucket name and prefix in the URL form.  The URL's scheme must be `s3`, and the host part corresponds to `bucket` while the path part does to `key_prefix`.  You may not specify `bucket_url` and either `bucket` or `key_prefix` at the same time.\n\n* `profile` (optional, defaults to the value of `AWS_PROFILE` unless `credentials` is specified)\n\n    Specifies the credentials profile name.\n\n* `region` (optional, defaults to the value of `AWS_REGION` environment variable)\n\n    Specifies the region of the endpoint.\n\n* `credentials` (optional)\n\n    * `credentials.aws_access_key_id` (required)\n    \n        Specifies the AWS access key.\n\n    * `credentials.aws_secret_access_key` (required)\n    \n        Specifies the AWS secret access key.\n\n* `max_object_size` (optional, defaults to unlimited)\n\n\tSpecifies the maximum size of an object put to S3.  This actually sets the size of the in-memory buffer used to hold the entire content sent from the client, as we have to calculate a MD5 sum for it before uploading there.\n\n* `readable` (optional, defaults to `true`)\n\n\tSpecifies whether to allow the client to fetch objects from S3.\n\n* `writable` (optional, defaults to `true`)\n\n\tSpecifies whether to allow the client to put objects to S3.\n\n* `listable` (optional, defaults to `true`)\n\n\tSpecifies whether to allow the client to list objects in S3.\n\n* `server_side_encryption` (optional, defaults to `\"none\"`)\n\n\tSpecifies which server-side encryption scheme is applied to store the objects.  Valid values are: `\"aes256\"` and `\"kms\"`.\n\n* `sse_customer_key` (required when `server_side_encryption` is set to `\"aes256\"`)\n\n\tSpecifies the base64-encoded encryption key.  As the cipher is AES256-CBC, the key must be 256-bits long (32 bytes)\n\n* `sse_kms_key_id` (required when `server_side_encryption` is est to `\"kms\"`)\n\n\tSpecifies the CMK ID used for the server-side encryption using KMS.\n\n* `keyboard_interactive_auth` (optional, defaults to `false`)\n\n    Enables keyboard interactive authentication if set to true.\n\n* `auth` (required)\n\n    Specifies the name of the authenticator.\n\n\n### Authenticator Settings\n\n```toml\n[auth.test]\ntype = \"inplace\"\n\n# authenticator specific settings follow\n```\n\n* `type` (required)\n\n    Specifies the authenticator implementation type.  Currently `\"inplace\"` is the only valid value.\n\n* `users` (required when `type` is `\"inplace\"`)\n\n    Contains user records as a dictionary.\n\n\n#### In-place authenticator\n\nIn-place authenticator reads the credentials directly embedded in the configuration file.  The user record looks like the following:\n\n```toml\n[auth.test]\ntype = \"inplace\"\n\n[auth.test.users.user0]\npassword = \"test\"\npublic_keys = \"\"\"\nssh-rsa AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nssh-rsa AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n\"\"\"\n\n[auth.test.users.user1]\npassword = \"test\"\npublic_keys = \"\"\"\nssh-rsa AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nssh-rsa AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n\"\"\"\n```\n\nOr \n\n```toml\n[auth.test]\ntype = \"inplace\"\n\n[auth.test.users]\nuser0 = { password=\"test\", public_keys=\"...\" }\nuser1 = { password=\"test\", public_keys=\"...\" }\n```\n\n* (key) (appears as `user0` or `user1` in the above example)\n\n    Specifies the name of the user.\n\n* `password` (optional)\n\n    Specifies the password in a clear-text form.\n\n* `public_keys` (optional)\n\n    Specifies the public keys authorized to use in authentication.  Multiple keys can be specified by delimiting them by newlines.\n\n"
  },
  {
    "path": "bucket.go",
    "content": "package main\n\nimport (\n\t\"crypto\"\n\t\"encoding/base64\"\n\t\"fmt\"\n\t\"strings\"\n\n\taws \"github.com/aws/aws-sdk-go/aws\"\n\taws_creds \"github.com/aws/aws-sdk-go/aws/credentials\"\n\taws_ec2_role_creds \"github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds\"\n\taws_ec2_meta \"github.com/aws/aws-sdk-go/aws/ec2metadata\"\n\taws_session \"github.com/aws/aws-sdk-go/aws/session\"\n\ts3 \"github.com/aws/aws-sdk-go/service/s3\"\n\t\"github.com/pkg/errors\"\n)\n\ntype ServerSideEncryptionType int\n\nconst (\n\tServerSideEncryptionTypeNone = iota\n\tServerSideEncryptionTypeAES256\n\tServerSideEncryptionTypeKMS\n)\n\nvar sseNameToEnumMap = map[string]ServerSideEncryptionType{\n\t\"\":       ServerSideEncryptionTypeNone,\n\t\"none\":   ServerSideEncryptionTypeNone,\n\t\"aes256\": ServerSideEncryptionTypeAES256,\n\t\"kms\":    ServerSideEncryptionTypeKMS,\n}\n\nfunc (v *ServerSideEncryptionType) UnmarshalText(text []byte) error {\n\t_v, ok := sseNameToEnumMap[strings.ToLower(string(text))]\n\tif !ok {\n\t\treturn fmt.Errorf(\"invalid value for ServerSideEncryption: %s\", string(text))\n\t}\n\t*v = _v\n\treturn nil\n}\n\ntype ServerSideEncryptionConfig struct {\n\tType           ServerSideEncryptionType\n\tCustomerKey    string\n\tCustomerKeyMD5 string\n\tKMSKeyId       string\n}\n\nfunc (cfg *ServerSideEncryptionConfig) CustomerAlgorithm() string {\n\tif cfg.Type == ServerSideEncryptionTypeAES256 {\n\t\treturn \"AES256\"\n\t} else {\n\t\treturn \"\"\n\t}\n}\n\ntype Perms struct {\n\tReadable bool\n\tWritable bool\n\tListable bool\n}\n\ntype S3Bucket struct {\n\tName                           string\n\tAWSConfig                      *aws.Config\n\tBucket                         string\n\tKeyPrefix                      Path\n\tMaxObjectSize                  int64\n\tUsers                          UserStore\n\tPerms                          Perms\n\tServerSideEncryption           ServerSideEncryptionConfig\n\tKeyboardInteractiveAuthEnabled bool\n}\n\ntype S3Buckets struct {\n\tBuckets         map[string]*S3Bucket\n\tUserToBucketMap map[string]*S3Bucket\n}\n\nfunc (s3bs *S3Buckets) Get(name string) *S3Bucket {\n\tb, _ := s3bs.Buckets[name]\n\treturn b\n}\n\nfunc (s3b *S3Bucket) S3(sess *aws_session.Session) *s3.S3 {\n\tawsCfg := s3b.AWSConfig\n\tif awsCfg.Credentials == nil {\n\t\tawsCfg = s3b.AWSConfig.WithCredentials(aws_creds.NewChainCredentials(\n\t\t\t[]aws_creds.Provider{\n\t\t\t\t&aws_ec2_role_creds.EC2RoleProvider{\n\t\t\t\t\tClient:       aws_ec2_meta.New(sess),\n\t\t\t\t\tExpiryWindow: 0,\n\t\t\t\t},\n\t\t\t\t&aws_creds.EnvProvider{},\n\t\t\t},\n\t\t))\n\t}\n\treturn s3.New(sess, awsCfg)\n}\n\nfunc buildS3Bucket(uStores UserStores, name string, bCfg *S3BucketConfig) (*S3Bucket, error) {\n\tawsCfg := aws.NewConfig()\n\tif bCfg.Credentials != nil {\n\t\tawsCfg = awsCfg.WithCredentials(\n\t\t\taws_creds.NewStaticCredentials(\n\t\t\t\tbCfg.Credentials.AWSAccessKeyID,\n\t\t\t\tbCfg.Credentials.AWSSecretAccessKey,\n\t\t\t\t\"\",\n\t\t\t),\n\t\t)\n\t} else if bCfg.Profile != \"\" {\n\t\tawsCfg = awsCfg.WithCredentials(\n\t\t\taws_creds.NewSharedCredentials(\n\t\t\t\t\"\", // TODO: assumes default\n\t\t\t\tbCfg.Profile,\n\t\t\t),\n\t\t)\n\t} else {\n\t\t// credentials are retrieved through EC2 metadata on runtime\n\t}\n\tif bCfg.Endpoint != \"\" {\n\t\tawsCfg = awsCfg.WithEndpoint(bCfg.Endpoint)\n\t}\n\tif bCfg.S3ForcePathStyle != nil {\n\t\tawsCfg = awsCfg.WithS3ForcePathStyle(*bCfg.S3ForcePathStyle)\n\t}\n\tif bCfg.DisableSSL != nil {\n\t\tawsCfg = awsCfg.WithDisableSSL(*bCfg.DisableSSL)\n\t}\n\tif bCfg.Region != \"\" {\n\t\tawsCfg = awsCfg.WithRegion(bCfg.Region)\n\t}\n\tusers, ok := uStores[bCfg.Auth]\n\tif !ok {\n\t\treturn nil, fmt.Errorf(\"no such auth config: %s\", bCfg.Auth)\n\t}\n\tkeyPrefix := SplitIntoPath(bCfg.KeyPrefix)\n\tif len(keyPrefix) > 0 && keyPrefix[0] == \"\" {\n\t\tkeyPrefix = keyPrefix[1:]\n\t}\n\tmaxObjectSize := int64(-1)\n\tif bCfg.MaxObjectSize != nil {\n\t\tmaxObjectSize = *bCfg.MaxObjectSize\n\t}\n\n\tvar customerKey []byte\n\tvar customerKeyMD5 string\n\tif bCfg.SSECustomerKey != \"\" {\n\t\tvar err error\n\t\tcustomerKey, err = base64.StdEncoding.DecodeString(bCfg.SSECustomerKey)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, `invalid base64-encoded string specified for \"sse_customer_key\"`)\n\t\t}\n\t\thasher := crypto.MD5.New()\n\t\thasher.Write(customerKey)\n\t\tcustomerKeyMD5 = base64.StdEncoding.EncodeToString(hasher.Sum([]byte{}))\n\t} else {\n\t\tcustomerKey = []byte{}\n\t}\n\treturn &S3Bucket{\n\t\tName:          name,\n\t\tAWSConfig:     awsCfg,\n\t\tBucket:        bCfg.Bucket,\n\t\tKeyPrefix:     keyPrefix,\n\t\tMaxObjectSize: maxObjectSize,\n\t\tUsers:         users,\n\t\tPerms: Perms{\n\t\t\tReadable: *bCfg.Readable,\n\t\t\tWritable: *bCfg.Writable,\n\t\t\tListable: *bCfg.Listable,\n\t\t},\n\t\tServerSideEncryption: ServerSideEncryptionConfig{\n\t\t\tType:           bCfg.ServerSideEncryption,\n\t\t\tCustomerKey:    string(customerKey),\n\t\t\tCustomerKeyMD5: customerKeyMD5,\n\t\t\tKMSKeyId:       bCfg.SSEKMSKeyId,\n\t\t},\n\t\tKeyboardInteractiveAuthEnabled: bCfg.KeyboardInteractiveAuthEnabled,\n\t}, nil\n}\n\nfunc NewS3BucketFromConfig(uStores UserStores, cfg *S3SFTPProxyConfig) (*S3Buckets, error) {\n\tbuckets := map[string]*S3Bucket{}\n\tuserToBucketMap := map[string]*S3Bucket{}\n\tfor name, bCfg := range cfg.Buckets {\n\t\tbucket, err := buildS3Bucket(uStores, name, bCfg)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"bucket config %s\", name)\n\t\t}\n\t\tfor _, user := range bucket.Users.Users {\n\t\t\t_bucket, ok := userToBucketMap[user.Name]\n\t\t\tif ok {\n\t\t\t\treturn nil, fmt.Errorf(`bucket config %s: user \"%s\" is already assigned to bucket config \"%s\"`, name, user.Name, _bucket.Name)\n\t\t\t}\n\t\t\tuserToBucketMap[user.Name] = bucket\n\t\t}\n\t\tbuckets[name] = bucket\n\t}\n\treturn &S3Buckets{\n\t\tBuckets:         buckets,\n\t\tUserToBucketMap: userToBucketMap,\n\t}, nil\n}\n"
  },
  {
    "path": "bucketio.go",
    "content": "package main\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"path\"\n\t\"sync\"\n\t\"time\"\n\n\taws \"github.com/aws/aws-sdk-go/aws\"\n\taws_session \"github.com/aws/aws-sdk-go/aws/session\"\n\taws_s3 \"github.com/aws/aws-sdk-go/service/s3\"\n\t\"github.com/pkg/sftp\"\n\t// s3crypto \"github.com/aws/aws-sdk-go/service/s3/s3crypto\"\n)\n\nvar aclPrivate = \"private\"\n\ntype ReadDeadlineSettable interface {\n\tSetReadDeadline(t time.Time) error\n}\n\ntype WriteDeadlineSettable interface {\n\tSetWriteDeadline(t time.Time) error\n}\n\nvar sseTypes = map[ServerSideEncryptionType]*string{\n\tServerSideEncryptionTypeKMS: aws.String(\"aws:kms\"),\n}\n\nfunc nilIfEmpty(s string) *string {\n\tif s == \"\" {\n\t\treturn nil\n\t} else {\n\t\treturn &s\n\t}\n}\n\ntype S3GetObjectOutputReader struct {\n\tCtx          context.Context\n\tGoo          *aws_s3.GetObjectOutput\n\tLog          DebugLogger\n\tLookback     int\n\tMinChunkSize int\n\tmtx          sync.Mutex\n\tspooled      []byte\n\tspoolOffset  int\n\tnoMore       bool\n}\n\nfunc (oor *S3GetObjectOutputReader) Close() error {\n\tif oor.Goo.Body != nil {\n\t\toor.Goo.Body.Close()\n\t\toor.Goo.Body = nil\n\t}\n\treturn nil\n}\n\nfunc (oor *S3GetObjectOutputReader) ReadAt(buf []byte, off int64) (int, error) {\n\toor.mtx.Lock()\n\tdefer oor.mtx.Unlock()\n\n\tF(oor.Log.Debug, \"len(buf)=%d, off=%d\", len(buf), off)\n\t_o, err := castInt64ToInt(off)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\tif _o < oor.spoolOffset {\n\t\treturn 0, fmt.Errorf(\"supplied position is out of range\")\n\t}\n\n\ts := _o - oor.spoolOffset\n\ti := 0\n\tr := len(buf)\n\tif s < len(oor.spooled) {\n\t\t// n = max(r, len(oor.spooled)-s)\n\t\tn := r\n\t\tif n > len(oor.spooled)-s {\n\t\t\tn = len(oor.spooled) - s\n\t\t}\n\t\tcopy(buf[i:i+n], oor.spooled[s:s+n])\n\t\ti += n\n\t\ts += n\n\t\tr -= n\n\t}\n\tif r == 0 {\n\t\treturn i, nil\n\t}\n\n\tif oor.noMore {\n\t\tif i == 0 {\n\t\t\treturn 0, io.EOF\n\t\t} else {\n\t\t\treturn i, nil\n\t\t}\n\t}\n\n\tF(oor.Log.Debug, \"s=%d, len(oor.spooled)=%d, oor.Lookback=%d\", s, len(oor.spooled), oor.Lookback)\n\tif s <= len(oor.spooled) && s >= oor.Lookback {\n\t\toor.spooled = oor.spooled[s-oor.Lookback:]\n\t\toor.spoolOffset += s - oor.Lookback\n\t\ts = oor.Lookback\n\t}\n\n\tvar e int\n\tif len(oor.spooled)+oor.MinChunkSize < s+r {\n\t\te = s + r\n\t} else {\n\t\te = len(oor.spooled) + oor.MinChunkSize\n\t}\n\n\tif cap(oor.spooled) < e {\n\t\tspooled := make([]byte, len(oor.spooled), e)\n\t\tcopy(spooled, oor.spooled)\n\t\toor.spooled = spooled\n\t}\n\n\ttype readResult struct {\n\t\tn   int\n\t\terr error\n\t}\n\n\tresultChan := make(chan readResult)\n\tgo func() {\n\t\tn, err := io.ReadFull(oor.Goo.Body, oor.spooled[len(oor.spooled):e])\n\t\tresultChan <- readResult{n, err}\n\t}()\n\tselect {\n\tcase <-oor.Ctx.Done():\n\t\toor.Goo.Body.(ReadDeadlineSettable).SetReadDeadline(time.Unix(1, 0))\n\t\toor.Log.Debug(\"canceled\")\n\t\treturn 0, fmt.Errorf(\"read operation canceled\")\n\tcase res := <-resultChan:\n\t\tif IsEOF(res.err) {\n\t\t\toor.noMore = true\n\t\t}\n\t\te = len(oor.spooled) + res.n\n\t\toor.spooled = oor.spooled[:e]\n\t\tif s < e {\n\t\t\tbe := e\n\t\t\tif be > s+r {\n\t\t\t\tbe = s + r\n\t\t\t}\n\t\t\tcopy(buf[i:], oor.spooled[s:be])\n\t\t\treturn be - s, nil\n\t\t} else {\n\t\t\treturn 0, io.EOF\n\t\t}\n\t}\n}\n\ntype S3PutObjectWriter struct {\n\tCtx                  context.Context\n\tBucket               string\n\tKey                  Path\n\tS3                   *aws_s3.S3\n\tServerSideEncryption *ServerSideEncryptionConfig\n\tLog                  interface {\n\t\tDebugLogger\n\t\tErrorLogger\n\t}\n\tMaxObjectSize    int64\n\tInfo             *PhantomObjectInfo\n\tPhantomObjectMap *PhantomObjectMap\n\tmtx              sync.Mutex\n\twriter           *BytesWriter\n}\n\nfunc (oow *S3PutObjectWriter) Close() error {\n\tF(oow.Log.Debug, \"S3PutObjectWriter.Close\")\n\toow.mtx.Lock()\n\tdefer oow.mtx.Unlock()\n\tphInfo := oow.Info.GetOne()\n\toow.PhantomObjectMap.RemoveByInfoPtr(oow.Info)\n\tkey := phInfo.Key.String()\n\tsse := oow.ServerSideEncryption\n\tF(oow.Log.Debug, \"PutObject(Bucket=%s, Key=%s, Sse=%v)\", oow.Bucket, key, sse)\n\t_, err := oow.S3.PutObject(\n\t\t&aws_s3.PutObjectInput{\n\t\t\tACL:                  &aclPrivate,\n\t\t\tBody:                 bytes.NewReader(oow.writer.Bytes()),\n\t\t\tBucket:               &oow.Bucket,\n\t\t\tKey:                  &key,\n\t\t\tServerSideEncryption: sseTypes[sse.Type],\n\t\t\tSSECustomerAlgorithm: nilIfEmpty(sse.CustomerAlgorithm()),\n\t\t\tSSECustomerKey:       nilIfEmpty(sse.CustomerKey),\n\t\t\tSSECustomerKeyMD5:    nilIfEmpty(sse.CustomerKeyMD5),\n\t\t\tSSEKMSKeyId:          nilIfEmpty(sse.KMSKeyId),\n\t\t},\n\t)\n\tif err != nil {\n\t\toow.Log.Debug(\"=> \", err)\n\t\tF(oow.Log.Error, \"failed to put object: %s\", err.Error())\n\t} else {\n\t\toow.Log.Debug(\"=> OK\")\n\t}\n\treturn nil\n}\n\nfunc (oow *S3PutObjectWriter) WriteAt(buf []byte, off int64) (int, error) {\n\toow.mtx.Lock()\n\tdefer oow.mtx.Unlock()\n\tif oow.MaxObjectSize >= 0 {\n\t\tif int64(len(buf))+off > oow.MaxObjectSize {\n\t\t\treturn 0, fmt.Errorf(\"file too large: maximum allowed size is %d bytes\", oow.MaxObjectSize)\n\t\t}\n\t}\n\tF(oow.Log.Debug, \"len(buf)=%d, off=%d\", len(buf), off)\n\tn, err := oow.writer.WriteAt(buf, off)\n\toow.Info.SetSize(oow.writer.Size())\n\treturn n, err\n}\n\ntype ObjectFileInfo struct {\n\t_Name         string\n\t_LastModified time.Time\n\t_Size         int64\n\t_Mode         os.FileMode\n}\n\nfunc (ofi *ObjectFileInfo) Name() string {\n\treturn ofi._Name\n}\n\nfunc (ofi *ObjectFileInfo) ModTime() time.Time {\n\treturn ofi._LastModified\n}\n\nfunc (ofi *ObjectFileInfo) Size() int64 {\n\treturn ofi._Size\n}\n\nfunc (ofi *ObjectFileInfo) Mode() os.FileMode {\n\treturn ofi._Mode\n}\n\nfunc (ofi *ObjectFileInfo) IsDir() bool {\n\treturn (ofi._Mode & os.ModeDir) != 0\n}\n\nfunc (ofi *ObjectFileInfo) Sys() interface{} {\n\treturn BuildFakeFileInfoSys()\n}\n\ntype S3ObjectLister struct {\n\tDebugLogger\n\tCtx              context.Context\n\tBucket           string\n\tPrefix           Path\n\tS3               *aws_s3.S3\n\tLookback         int\n\tPhantomObjectMap *PhantomObjectMap\n\tspoolOffset      int\n\tspooled          []os.FileInfo\n\tcontinuation     *string\n\tnoMore           bool\n}\n\nfunc aclToMode(owner *aws_s3.Owner, grants []*aws_s3.Grant) os.FileMode {\n\tvar v os.FileMode\n\tfor _, g := range grants {\n\t\tif g.Grantee != nil {\n\t\t\tif g.Grantee.ID != nil && *g.Grantee.ID == *owner.ID {\n\t\t\t\tswitch *g.Permission {\n\t\t\t\tcase \"READ\":\n\t\t\t\t\tv |= 0400\n\t\t\t\tcase \"WRITE\":\n\t\t\t\t\tv |= 0200\n\t\t\t\tcase \"FULL_CONTROL\":\n\t\t\t\t\tv |= 0600\n\t\t\t\t}\n\t\t\t} else if g.Grantee.URI != nil {\n\t\t\t\tswitch *g.Grantee.URI {\n\t\t\t\tcase \"http://acs.amazonaws.com/groups/global/AuthenticatedUsers\":\n\t\t\t\t\tswitch *g.Permission {\n\t\t\t\t\tcase \"READ\":\n\t\t\t\t\t\tv |= 0440\n\t\t\t\t\tcase \"WRITE\":\n\t\t\t\t\t\tv |= 0220\n\t\t\t\t\tcase \"FULL_CONTROL\":\n\t\t\t\t\t\tv |= 0660\n\t\t\t\t\t}\n\t\t\t\tcase \"http://acs.amazonaws.com/groups/global/AllUsers\":\n\t\t\t\t\tswitch *g.Permission {\n\t\t\t\t\tcase \"READ\":\n\t\t\t\t\t\tv |= 0444\n\t\t\t\t\tcase \"WRITE\":\n\t\t\t\t\t\tv |= 0222\n\t\t\t\t\tcase \"FULL_CONTROL\":\n\t\t\t\t\t\tv |= 0666\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\treturn v\n}\n\nfunc (sol *S3ObjectLister) ListAt(result []os.FileInfo, o int64) (int, error) {\n\t_o, err := castInt64ToInt(o)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\tif _o < sol.spoolOffset {\n\t\treturn 0, fmt.Errorf(\"supplied position is out of range\")\n\t}\n\n\ts := _o - sol.spoolOffset\n\ti := 0\n\tif s < len(sol.spooled) {\n\t\tn := len(result)\n\t\tif n > len(sol.spooled)-s {\n\t\t\tn = len(sol.spooled) - s\n\t\t}\n\t\tcopy(result[i:i+n], sol.spooled[s:s+n])\n\t\ti += n\n\t\ts = len(sol.spooled)\n\t}\n\n\tif i >= len(result) {\n\t\treturn i, nil\n\t}\n\n\tif sol.noMore {\n\t\tif i == 0 {\n\t\t\treturn 0, io.EOF\n\t\t} else {\n\t\t\treturn i, nil\n\t\t}\n\t}\n\n\tif s <= len(sol.spooled) && s >= sol.Lookback {\n\t\tsol.spooled = sol.spooled[s-sol.Lookback:]\n\t\tsol.spoolOffset += s - sol.Lookback\n\t\ts = sol.Lookback\n\t}\n\n\tif sol.continuation == nil {\n\t\tsol.spooled = append(sol.spooled, &ObjectFileInfo{\n\t\t\t_Name:         \".\",\n\t\t\t_LastModified: time.Unix(1, 0),\n\t\t\t_Size:         0,\n\t\t\t_Mode:         0755 | os.ModeDir,\n\t\t})\n\t\tsol.spooled = append(sol.spooled, &ObjectFileInfo{\n\t\t\t_Name:         \"..\",\n\t\t\t_LastModified: time.Unix(1, 0),\n\t\t\t_Size:         0,\n\t\t\t_Mode:         0755 | os.ModeDir,\n\t\t})\n\n\t\tphObjs := sol.PhantomObjectMap.List(sol.Prefix)\n\t\tfor _, phInfo := range phObjs {\n\t\t\t_phInfo := phInfo.GetOne()\n\t\t\tsol.spooled = append(sol.spooled, &ObjectFileInfo{\n\t\t\t\t_Name:         _phInfo.Key.Base(),\n\t\t\t\t_LastModified: _phInfo.LastModified,\n\t\t\t\t_Size:         _phInfo.Size,\n\t\t\t\t_Mode:         0600, // TODO\n\t\t\t})\n\t\t}\n\t}\n\n\tprefix := sol.Prefix.String()\n\tif prefix != \"\" {\n\t\tprefix += \"/\"\n\t}\n\tF(sol.Debug, \"ListObjectsV2WithContext(Bucket=%s, Prefix=%s, Continuation=%v)\", sol.Bucket, prefix, sol.continuation)\n\tout, err := sol.S3.ListObjectsV2WithContext(\n\t\tsol.Ctx,\n\t\t&aws_s3.ListObjectsV2Input{\n\t\t\tBucket:            &sol.Bucket,\n\t\t\tPrefix:            &prefix,\n\t\t\tMaxKeys:           aws.Int64(10000),\n\t\t\tDelimiter:         aws.String(\"/\"),\n\t\t\tContinuationToken: sol.continuation,\n\t\t},\n\t)\n\tif err != nil {\n\t\tsol.Debug(\"=> \", err)\n\t\treturn i, err\n\t}\n\tF(sol.Debug, \"=> { CommonPrefixes=len(%d), Contents=len(%d) }\", len(out.CommonPrefixes), len(out.Contents))\n\n\tif sol.continuation == nil {\n\t\tfor _, cPfx := range out.CommonPrefixes {\n\t\t\tsol.spooled = append(sol.spooled, &ObjectFileInfo{\n\t\t\t\t_Name:         path.Base(*cPfx.Prefix),\n\t\t\t\t_LastModified: time.Unix(1, 0),\n\t\t\t\t_Size:         0,\n\t\t\t\t_Mode:         0755 | os.ModeDir,\n\t\t\t})\n\t\t}\n\t}\n\tfor _, obj := range out.Contents {\n\t\t// if *obj.Key == sol.Prefix {\n\t\t// \tcontinue\n\t\t// }\n\t\tsol.spooled = append(sol.spooled, &ObjectFileInfo{\n\t\t\t_Name:         path.Base(*obj.Key),\n\t\t\t_LastModified: *obj.LastModified,\n\t\t\t_Size:         *obj.Size,\n\t\t\t_Mode:         0644,\n\t\t})\n\t}\n\tsol.continuation = out.NextContinuationToken\n\tif out.NextContinuationToken == nil {\n\t\tsol.noMore = true\n\t}\n\n\tvar n int\n\tif len(sol.spooled)-s > len(result)-i {\n\t\tn = len(result) - i\n\t} else {\n\t\tn = len(sol.spooled) - s\n\t\tif sol.noMore {\n\t\t\terr = io.EOF\n\t\t}\n\t}\n\n\tcopy(result[i:i+n], sol.spooled[s:s+n])\n\treturn i + n, err\n}\n\ntype S3ObjectStat struct {\n\tDebugLogger\n\tCtx              context.Context\n\tBucket           string\n\tKey              Path\n\tRoot             bool\n\tS3               *aws_s3.S3\n\tPhantomObjectMap *PhantomObjectMap\n}\n\nfunc (sos *S3ObjectStat) ListAt(result []os.FileInfo, o int64) (int, error) {\n\tF(sos.Debug, \"S3ObjectStat.ListAt: len(result)=%d offset=%d\", len(result), o)\n\t_o, err := castInt64ToInt(o)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\tif len(result) == 0 {\n\t\treturn 0, nil\n\t}\n\n\tif _o > 0 {\n\t\treturn 0, fmt.Errorf(\"supplied position is out of range\")\n\t}\n\n\tif sos.Key.IsRoot() {\n\t\tresult[0] = &ObjectFileInfo{\n\t\t\t_Name:         \"/\",\n\t\t\t_LastModified: time.Time{},\n\t\t\t_Size:         0,\n\t\t\t_Mode:         0755 | os.ModeDir,\n\t\t}\n\t} else {\n\t\tphInfo := sos.PhantomObjectMap.Get(sos.Key)\n\t\tif phInfo != nil {\n\t\t\t_phInfo := phInfo.GetOne()\n\t\t\tresult[0] = &ObjectFileInfo{\n\t\t\t\t_Name:         _phInfo.Key.Base(),\n\t\t\t\t_LastModified: _phInfo.LastModified,\n\t\t\t\t_Size:         _phInfo.Size,\n\t\t\t\t_Mode:         0600, // TODO\n\t\t\t}\n\t\t} else {\n\t\t\tkey := sos.Key.String()\n\t\t\tF(sos.Debug, \"GetObjectAclWithContext(Bucket=%s, Key=%s)\", sos.Bucket, key)\n\t\t\tout, err := sos.S3.GetObjectAclWithContext(\n\t\t\t\tsos.Ctx,\n\t\t\t\t&aws_s3.GetObjectAclInput{\n\t\t\t\t\tBucket: &sos.Bucket,\n\t\t\t\t\tKey:    &key,\n\t\t\t\t},\n\t\t\t)\n\t\t\tif err == nil {\n\t\t\t\tF(sos.Debug, \"=> %v\", out)\n\t\t\t\tF(sos.Debug, \"HeadObjectWithContext(Bucket=%s, Key=%s)\", sos.Bucket, key)\n\t\t\t\theadOut, err := sos.S3.HeadObjectWithContext(\n\t\t\t\t\tsos.Ctx,\n\t\t\t\t\t&aws_s3.HeadObjectInput{\n\t\t\t\t\t\tBucket: &sos.Bucket,\n\t\t\t\t\t\tKey:    &key,\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t\tobjInfo := ObjectFileInfo{\n\t\t\t\t\t_Name: sos.Key.Base(),\n\t\t\t\t\t_Mode: aclToMode(out.Owner, out.Grants),\n\t\t\t\t}\n\t\t\t\tif err == nil {\n\t\t\t\t\tF(sos.Debug, \"=> { ContentLength=%d, LastModified=%v }\", *headOut.ContentLength, *headOut.LastModified)\n\t\t\t\t\tobjInfo._Size = *headOut.ContentLength\n\t\t\t\t\tobjInfo._LastModified = *headOut.LastModified\n\t\t\t\t} else {\n\t\t\t\t\tsos.Debug(\"=> \", err)\n\t\t\t\t}\n\t\t\t\tresult[0] = &objInfo\n\t\t\t} else {\n\t\t\t\tsos.Debug(\"=> \", err)\n\t\t\t\tF(sos.Debug, \"ListObjectsV2WithContext(Bucket=%s, Prefix=%s)\", sos.Bucket, key)\n\t\t\t\tout, err := sos.S3.ListObjectsV2WithContext(\n\t\t\t\t\tsos.Ctx,\n\t\t\t\t\t&aws_s3.ListObjectsV2Input{\n\t\t\t\t\t\tBucket:    &sos.Bucket,\n\t\t\t\t\t\tPrefix:    &key,\n\t\t\t\t\t\tMaxKeys:   aws.Int64(10000),\n\t\t\t\t\t\tDelimiter: aws.String(\"/\"),\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t\tif err != nil || (!sos.Root && len(out.CommonPrefixes) == 0) {\n\t\t\t\t\tsos.Debug(\"=> \", err)\n\t\t\t\t\treturn 0, os.ErrNotExist\n\t\t\t\t}\n\t\t\t\tF(sos.Debug, \"=> { CommonPrefixes=len(%d), Contents=len(%d) }\", len(out.CommonPrefixes), len(out.Contents))\n\t\t\t\tresult[0] = &ObjectFileInfo{\n\t\t\t\t\t_Name:         sos.Key.Base(),\n\t\t\t\t\t_LastModified: time.Time{},\n\t\t\t\t\t_Size:         0,\n\t\t\t\t\t_Mode:         0755 | os.ModeDir,\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\treturn 1, nil\n}\n\ntype S3BucketIO struct {\n\tCtx                      context.Context\n\tBucket                   *S3Bucket\n\tReaderLookbackBufferSize int\n\tReaderMinChunkSize       int\n\tListerLookbackBufferSize int\n\tPhantomObjectMap         *PhantomObjectMap\n\tPerms                    Perms\n\tServerSideEncryption     *ServerSideEncryptionConfig\n\tNow                      func() time.Time\n\tLog                      interface {\n\t\tErrorLogger\n\t\tDebugLogger\n\t}\n}\n\nfunc buildKey(s3b *S3Bucket, path string) Path {\n\treturn s3b.KeyPrefix.Join(SplitIntoPath(path))\n}\n\nfunc buildPath(s3b *S3Bucket, key string) (string, bool) {\n\t_key := SplitIntoPath(key)\n\tif !_key.IsPrefixed(s3b.KeyPrefix) {\n\t\treturn \"\", false\n\t}\n\treturn \"/\" + _key[len(s3b.KeyPrefix):].String(), true\n}\n\nfunc (s3io *S3BucketIO) Fileread(req *sftp.Request) (io.ReaderAt, error) {\n\tif !s3io.Perms.Readable {\n\t\treturn nil, fmt.Errorf(\"read operation not allowed as per configuration\")\n\t}\n\tsess, err := aws_session.NewSession()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\ts3 := s3io.Bucket.S3(sess)\n\tkey := buildKey(s3io.Bucket, req.Filepath)\n\n\tphInfo := s3io.PhantomObjectMap.Get(key)\n\tif phInfo != nil {\n\t\treturn bytes.NewReader(phInfo.Opaque.(*S3PutObjectWriter).writer.Bytes()), nil\n\t}\n\n\tkeyStr := key.String()\n\tctx := combineContext(s3io.Ctx, req.Context())\n\tF(s3io.Log.Debug, \"GetObject(Bucket=%s, Key=%s)\", s3io.Bucket.Bucket, keyStr)\n\tsse := s3io.ServerSideEncryption\n\tgoo, err := s3.GetObjectWithContext(\n\t\tctx,\n\t\t&aws_s3.GetObjectInput{\n\t\t\tBucket:               &s3io.Bucket.Bucket,\n\t\t\tKey:                  &keyStr,\n\t\t\tSSECustomerAlgorithm: nilIfEmpty(sse.CustomerAlgorithm()),\n\t\t\tSSECustomerKey:       nilIfEmpty(sse.CustomerKey),\n\t\t\tSSECustomerKeyMD5:    nilIfEmpty(sse.CustomerKeyMD5),\n\t\t},\n\t)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn &S3GetObjectOutputReader{\n\t\tCtx:          ctx,\n\t\tGoo:          goo,\n\t\tLog:          s3io.Log,\n\t\tLookback:     s3io.ReaderLookbackBufferSize,\n\t\tMinChunkSize: s3io.ReaderMinChunkSize,\n\t}, nil\n}\n\nfunc (s3io *S3BucketIO) Filewrite(req *sftp.Request) (io.WriterAt, error) {\n\tif !s3io.Perms.Writable {\n\t\treturn nil, fmt.Errorf(\"write operation not allowed as per configuration\")\n\t}\n\tsess, err := aws_session.NewSession()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tmaxObjectSize := s3io.Bucket.MaxObjectSize\n\tif maxObjectSize < 0 {\n\t\tmaxObjectSize = int64(^uint(0) >> 1)\n\t}\n\tkey := buildKey(s3io.Bucket, req.Filepath)\n\tinfo := &PhantomObjectInfo{\n\t\tKey:          key,\n\t\tSize:         0,\n\t\tLastModified: s3io.Now(),\n\t}\n\tF(s3io.Log.Debug, \"S3PutObjectWriter.New(key=%s)\", key)\n\toow := &S3PutObjectWriter{\n\t\tCtx:                  combineContext(s3io.Ctx, req.Context()),\n\t\tBucket:               s3io.Bucket.Bucket,\n\t\tKey:                  key,\n\t\tS3:                   s3io.Bucket.S3(sess),\n\t\tServerSideEncryption: s3io.ServerSideEncryption,\n\t\tLog:                  s3io.Log,\n\t\tMaxObjectSize:        maxObjectSize,\n\t\tPhantomObjectMap:     s3io.PhantomObjectMap,\n\t\tInfo:                 info,\n\t\twriter:               NewBytesWriter(),\n\t}\n\tinfo.Opaque = oow\n\ts3io.PhantomObjectMap.Add(info)\n\treturn oow, nil\n}\n\nfunc (s3io *S3BucketIO) Filecmd(req *sftp.Request) error {\n\tswitch req.Method {\n\tcase \"Rename\":\n\t\tif !s3io.Perms.Writable {\n\t\t\treturn fmt.Errorf(\"write operation not allowed as per configuration\")\n\t\t}\n\t\tsrc := buildKey(s3io.Bucket, req.Filepath)\n\t\tdest := buildKey(s3io.Bucket, req.Target)\n\t\tif s3io.PhantomObjectMap.Rename(src, dest) {\n\t\t\treturn nil\n\t\t}\n\t\tsess, err := aws_session.NewSession()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tsrcStr := src.String()\n\t\tdestStr := dest.String()\n\t\tcopySource := s3io.Bucket.Bucket + \"/\" + srcStr\n\t\tsse := s3io.ServerSideEncryption\n\t\tF(s3io.Log.Debug, \"CopyObject(Bucket=%s, Key=%s, CopySource=%s, Sse=%v)\", s3io.Bucket.Bucket, destStr, copySource, sse.Type)\n\t\t_, err = s3io.Bucket.S3(sess).CopyObjectWithContext(\n\t\t\tcombineContext(s3io.Ctx, req.Context()),\n\t\t\t&aws_s3.CopyObjectInput{\n\t\t\t\tACL:                  &aclPrivate,\n\t\t\t\tBucket:               &s3io.Bucket.Bucket,\n\t\t\t\tCopySource:           &copySource,\n\t\t\t\tKey:                  &destStr,\n\t\t\t\tServerSideEncryption: sseTypes[sse.Type],\n\t\t\t\tSSECustomerAlgorithm: nilIfEmpty(sse.CustomerAlgorithm()),\n\t\t\t\tSSECustomerKey:       nilIfEmpty(sse.CustomerKey),\n\t\t\t\tSSECustomerKeyMD5:    nilIfEmpty(sse.CustomerKeyMD5),\n\t\t\t\tSSEKMSKeyId:          nilIfEmpty(sse.KMSKeyId),\n\t\t\t},\n\t\t)\n\t\tif err != nil {\n\t\t\ts3io.Log.Debug(\"=> \", err)\n\t\t\treturn err\n\t\t}\n\t\tF(s3io.Log.Debug, \"DeleteObject(Bucket=%s, Key=%s)\", s3io.Bucket.Bucket, srcStr)\n\t\t_, err = s3io.Bucket.S3(sess).DeleteObjectWithContext(\n\t\t\tcombineContext(s3io.Ctx, req.Context()),\n\t\t\t&aws_s3.DeleteObjectInput{\n\t\t\t\tBucket: &s3io.Bucket.Bucket,\n\t\t\t\tKey:    &srcStr,\n\t\t\t},\n\t\t)\n\t\tif err != nil {\n\t\t\ts3io.Log.Debug(\"=> \", err)\n\t\t\treturn err\n\t\t}\n\tcase \"Remove\":\n\t\tif !s3io.Perms.Writable {\n\t\t\treturn fmt.Errorf(\"write operation not allowed as per configuration\")\n\t\t}\n\t\tkey := buildKey(s3io.Bucket, req.Filepath)\n\t\tif s3io.PhantomObjectMap.Remove(key) != nil {\n\t\t\treturn nil\n\t\t}\n\t\tsess, err := aws_session.NewSession()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tkeyStr := key.String()\n\t\tF(s3io.Log.Debug, \"DeleteObject(Bucket=%s, Key=%s)\", s3io.Bucket.Bucket, key)\n\t\t_, err = s3io.Bucket.S3(sess).DeleteObjectWithContext(\n\t\t\tcombineContext(s3io.Ctx, req.Context()),\n\t\t\t&aws_s3.DeleteObjectInput{\n\t\t\t\tBucket: &s3io.Bucket.Bucket,\n\t\t\t\tKey:    &keyStr,\n\t\t\t},\n\t\t)\n\t\tif err != nil {\n\t\t\ts3io.Log.Debug(\"=> \", err)\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc (s3io *S3BucketIO) Filelist(req *sftp.Request) (sftp.ListerAt, error) {\n\tsess, err := aws_session.NewSession()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tswitch req.Method {\n\tcase \"Stat\", \"ReadLink\":\n\t\tif !s3io.Perms.Readable && !s3io.Perms.Listable {\n\t\t\treturn nil, fmt.Errorf(\"stat operation not allowed as per configuration\")\n\t\t}\n\t\tkey := buildKey(s3io.Bucket, req.Filepath)\n\t\treturn &S3ObjectStat{\n\t\t\tDebugLogger:      s3io.Log,\n\t\t\tCtx:              combineContext(s3io.Ctx, req.Context()),\n\t\t\tBucket:           s3io.Bucket.Bucket,\n\t\t\tRoot:             key.Equal(s3io.Bucket.KeyPrefix),\n\t\t\tKey:              key,\n\t\t\tS3:               s3io.Bucket.S3(sess),\n\t\t\tPhantomObjectMap: s3io.PhantomObjectMap,\n\t\t}, nil\n\tcase \"List\":\n\t\tif !s3io.Perms.Listable {\n\t\t\treturn nil, fmt.Errorf(\"listing operation not allowed as per configuration\")\n\t\t}\n\t\treturn &S3ObjectLister{\n\t\t\tDebugLogger:      s3io.Log,\n\t\t\tCtx:              combineContext(s3io.Ctx, req.Context()),\n\t\t\tBucket:           s3io.Bucket.Bucket,\n\t\t\tPrefix:           buildKey(s3io.Bucket, req.Filepath),\n\t\t\tS3:               s3io.Bucket.S3(sess),\n\t\t\tLookback:         s3io.ListerLookbackBufferSize,\n\t\t\tPhantomObjectMap: s3io.PhantomObjectMap,\n\t\t}, nil\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"unsupported method: %s\", req.Method)\n\t}\n}\n"
  },
  {
    "path": "config.go",
    "content": "package main\n\nimport (\n\t\"fmt\"\n\t\"github.com/BurntSushi/toml\"\n\t\"github.com/pkg/errors\"\n\t\"io/ioutil\"\n\t\"net/url\"\n)\n\nvar (\n\tminReaderLookbackBufferSize = 1048576\n\tminReaderMinChunkSize       = 262144\n\tminListerLookbackBufferSize = 100\n\tvTrue                       = true\n)\n\ntype URL struct {\n\t*url.URL\n}\n\nfunc (u *URL) UnmarshalText(text []byte) (err error) {\n\tu.URL, err = url.Parse(string(text))\n\treturn\n}\n\ntype AWSCredentialsConfig struct {\n\tAWSAccessKeyID     string `toml:\"aws_access_key_id\"`\n\tAWSSecretAccessKey string `toml:\"aws_secret_access_key\"`\n}\n\ntype S3BucketConfig struct {\n\tProfile                        string                   `toml:\"profile\"`\n\tCredentials                    *AWSCredentialsConfig    `toml:\"credentials\"`\n\tRegion                         string                   `toml:\"region\"`\n\tEndpoint                       string                   `toml:\"endpoint\"`\n\tDisableSSL                     *bool                    `toml:\"disable_ssl\"`\n\tS3ForcePathStyle               *bool                    `toml:\"s3_force_path_style\"`\n\tBucket                         string                   `toml:\"bucket\"`\n\tKeyPrefix                      string                   `toml:\"key_prefix\"`\n\tBucketUrl                      *URL                     `toml:\"bucket_url\"`\n\tAuth                           string                   `toml:\"auth\"`\n\tMaxObjectSize                  *int64                   `toml:\"max_object_size\"`\n\tReadable                       *bool                    `toml:\"readble\"`\n\tWritable                       *bool                    `toml:\"writable\"`\n\tListable                       *bool                    `toml:\"listable\"`\n\tServerSideEncryption           ServerSideEncryptionType `toml:\"server_side_encryption\"`\n\tSSECustomerKey                 string                   `toml:\"sse_customer_key\"`\n\tSSEKMSKeyId                    string                   `toml:\"sse_kms_key_id\"`\n\tKeyboardInteractiveAuthEnabled bool                     `toml:\"keyboard_interactive_auth\"`\n}\n\ntype AuthUser struct {\n\tPassword      string `toml:\"password\"`\n\tPublicKeys    string `toml:\"public_keys\"`\n\tPublicKeyFile string `toml:\"public_key_file\"`\n}\n\ntype AuthConfig struct {\n\tType       string              `toml:\"type\"`\n\tUserDBFile string              `toml:\"user_db_file\"`\n\tUsers      map[string]AuthUser `toml:\"users\"`\n}\n\ntype S3SFTPProxyConfig struct {\n\tBind                     string                     `toml:\"bind\"`\n\tHostKeyFile              string                     `toml:\"host_key_file\"`\n\tBanner                   string                     `toml:\"banner\"`\n\tReaderLookbackBufferSize *int                       `toml:\"reader_lookback_buffer_size\"`\n\tReaderMinChunkSize       *int                       `toml:\"reader_min_chunk_size\"`\n\tListerLookbackBufferSize *int                       `toml:\"lister_lookback_buffer_size\"`\n\tBuckets                  map[string]*S3BucketConfig `toml:\"buckets\"`\n\tAuthConfigs              map[string]*AuthConfig     `toml:\"auth\"`\n}\n\nfunc validateAndFixupBucketConfig(bCfg *S3BucketConfig) error {\n\tif bCfg.Profile != \"\" {\n\t\tif bCfg.Credentials != nil {\n\t\t\treturn fmt.Errorf(\"no credentials may be specified if profile is given\")\n\t\t}\n\t}\n\tif bCfg.BucketUrl != nil {\n\t\tif bCfg.Bucket != \"\" {\n\t\t\treturn fmt.Errorf(\"bucket may not be specified if bucket_url is given\")\n\t\t}\n\t\tif bCfg.KeyPrefix != \"\" {\n\t\t\treturn fmt.Errorf(\"root path may not be specified if bucket_url is given\")\n\t\t}\n\t\tif bCfg.BucketUrl.Host == \"\" {\n\t\t\treturn fmt.Errorf(\"bucket name is empty\")\n\t\t}\n\t\tif bCfg.BucketUrl.Scheme != \"s3\" {\n\t\t\treturn fmt.Errorf(\"bucket URL scheme must be \\\"s3\\\"\")\n\t\t}\n\t\tbCfg.Bucket = bCfg.BucketUrl.Host\n\t\tbCfg.KeyPrefix = bCfg.BucketUrl.Path\n\t} else {\n\t\tif bCfg.Bucket == \"\" {\n\t\t\treturn fmt.Errorf(\"bucket name is empty\")\n\t\t}\n\t}\n\tif bCfg.Auth == \"\" {\n\t\treturn fmt.Errorf(\"auth is not specified\")\n\t}\n\tif bCfg.Readable == nil {\n\t\tbCfg.Readable = &vTrue\n\t}\n\tif bCfg.Writable == nil {\n\t\tbCfg.Writable = &vTrue\n\t}\n\tif bCfg.Listable == nil {\n\t\tbCfg.Listable = &vTrue\n\t}\n\treturn nil\n}\n\nfunc validateAndFixupAuthConfigInplace(aCfg *AuthConfig) error {\n\tif aCfg.UserDBFile != \"\" {\n\t\treturn fmt.Errorf(`user_db_file may not be specified when auth type is \"inplace\"`)\n\t}\n\tif aCfg.Users == nil || len(aCfg.Users) == 0 {\n\t\tfmt.Printf(\"%#v\\n\", aCfg.Users)\n\t\treturn fmt.Errorf(`no \"users\" present`)\n\t}\n\treturn nil\n}\n\nfunc validateAndFixupAuthConfig(aCfg *AuthConfig) error {\n\tswitch aCfg.Type {\n\tcase \"inplace\":\n\t\treturn validateAndFixupAuthConfigInplace(aCfg)\n\tdefault:\n\t\treturn fmt.Errorf(\"unknown auth type: %s\", aCfg.Type)\n\t}\n}\n\nfunc ReadConfig(tomlStr string) (*S3SFTPProxyConfig, error) {\n\tcfg := &S3SFTPProxyConfig{\n\t\tBuckets:     map[string]*S3BucketConfig{},\n\t\tAuthConfigs: map[string]*AuthConfig{},\n\t}\n\n\t_, err := toml.Decode(tomlStr, cfg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif len(cfg.Buckets) == 0 {\n\t\treturn nil, fmt.Errorf(\"no bucket configs are present\")\n\t}\n\n\tif len(cfg.AuthConfigs) == 0 {\n\t\treturn nil, fmt.Errorf(\"no auth configs are present\")\n\t}\n\n\tif cfg.HostKeyFile == \"\" {\n\t\treturn nil, fmt.Errorf(\"no host key file is specified\")\n\t}\n\n\tif len(cfg.Banner) > 0 && cfg.Banner[len(cfg.Banner)-1] != '\\n' {\n\t\tcfg.Banner += \"\\n\"\n\t}\n\n\tif cfg.ReaderLookbackBufferSize == nil {\n\t\tcfg.ReaderLookbackBufferSize = &minReaderLookbackBufferSize\n\t} else if *cfg.ReaderLookbackBufferSize < minReaderLookbackBufferSize {\n\t\treturn nil, fmt.Errorf(\"reader_lookback_buffer_size must be equal to or greater than %d\", minReaderMinChunkSize)\n\t}\n\n\tif cfg.ReaderMinChunkSize == nil {\n\t\tcfg.ReaderMinChunkSize = &minReaderMinChunkSize\n\t} else if *cfg.ReaderMinChunkSize < minReaderMinChunkSize {\n\t\treturn nil, fmt.Errorf(\"reader_min_chunk_size must be equal to or greater than %d\", minReaderMinChunkSize)\n\t}\n\n\tif cfg.ListerLookbackBufferSize == nil {\n\t\tcfg.ListerLookbackBufferSize = &minListerLookbackBufferSize\n\t} else if *cfg.ListerLookbackBufferSize < minListerLookbackBufferSize {\n\t\treturn nil, fmt.Errorf(\"lister_lookback_buffer_size must be equal to or greater than %d\", minListerLookbackBufferSize)\n\t}\n\n\tfor name, bCfg := range cfg.Buckets {\n\t\terr := validateAndFixupBucketConfig(bCfg)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, `bucket config \"%s\"`, name)\n\t\t}\n\t}\n\n\tfor name, aCfg := range cfg.AuthConfigs {\n\t\terr := validateAndFixupAuthConfig(aCfg)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, `auth config \"%s\"`, name)\n\t\t}\n\t}\n\n\treturn cfg, err\n}\n\nfunc ReadConfigFromFile(tomlFile string) (*S3SFTPProxyConfig, error) {\n\ttomlStr, err := ioutil.ReadFile(tomlFile)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"failed to open %s\", tomlFile)\n\t}\n\n\tcfg, err := ReadConfig(string(tomlStr))\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"failed to parse %s\", tomlFile)\n\t}\n\n\treturn cfg, nil\n}\n"
  },
  {
    "path": "fakee_unix.go",
    "content": "// +build !windows\n\npackage main\n\nimport (\n\t\"syscall\"\n)\n\nfunc BuildFakeFileInfoSys() interface{} {\n\treturn &syscall.Stat_t{Uid: 65534, Gid: 65534}\n}\n"
  },
  {
    "path": "fakee_windows.go",
    "content": "// +build windows\n\npackage main\n\nimport \"syscall\"\n\nfunc BuildFakeFileInfoSys() interface{} {\n\treturn syscall.Win32FileAttributeData{}\n}\n"
  },
  {
    "path": "io.go",
    "content": "package main\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"unsafe\"\n)\n\ntype BytesWriter struct {\n\tbuf []byte\n\tpos int\n}\n\nfunc NewBytesWriter() *BytesWriter {\n\treturn &BytesWriter{\n\t\tbuf: []byte{},\n\t}\n}\n\nfunc castInt64ToInt(n int64) (int, error) {\n\tif unsafe.Sizeof(n) == unsafe.Sizeof(int(0)) {\n\t\treturn int(n), nil\n\t} else {\n\t\t_n := int(n)\n\t\tif int64(_n) < n {\n\t\t\treturn -1, fmt.Errorf(\"integer overflow detected when converting %#v to int\", n)\n\t\t}\n\t\treturn _n, nil\n\t}\n\n}\n\nfunc (bw *BytesWriter) Close() error {\n\treturn nil\n}\n\n// Resize the buffer capacity so the new size is at least the value of newCap.\nfunc (bw *BytesWriter) grow(newCap int) {\n\tif cap(bw.buf) >= newCap {\n\t\treturn\n\t}\n\ti := cap(bw.buf)\n\tif i < 2 {\n\t\ti = 2\n\t}\n\tfor i < newCap {\n\t\ti = i + i/2\n\t\tif i < cap(bw.buf) {\n\t\t\tpanic(\"allocation failure\")\n\t\t}\n\t}\n\tnewBuf := make([]byte, len(bw.buf), i)\n\tcopy(newBuf, bw.buf)\n\tbw.buf = newBuf\n}\n\nfunc (bw *BytesWriter) Truncate(n int64) error {\n\t_n, err := castInt64ToInt(n)\n\tif err != nil {\n\t\treturn err\n\t}\n\tbw.buf = bw.buf[0:_n]\n\tif bw.pos > _n {\n\t\tbw.pos = _n\n\t}\n\treturn nil\n}\n\nfunc (bw *BytesWriter) Seek(offset int64, whence int) (int64, error) {\n\t_o, err := castInt64ToInt(offset)\n\tif err != nil {\n\t\treturn -1, err\n\t}\n\tvar newPos int\n\tswitch whence {\n\tcase 0:\n\t\tnewPos = _o\n\tcase 1:\n\t\tnewPos = bw.pos + _o\n\tcase 2:\n\t\tnewPos = len(bw.buf) + _o\n\t}\n\tif newPos < len(bw.buf) {\n\t\tbw.grow(newPos)\n\t\tbw.buf = bw.buf[0:newPos]\n\t}\n\tbw.pos = newPos\n\treturn int64(newPos), nil\n}\n\nfunc (bw *BytesWriter) Write(p []byte) (n int, err error) {\n\tbw.grow(bw.pos + len(p))\n\tcopy(bw.buf[bw.pos:bw.pos+len(p)], p)\n\treturn len(p), nil\n}\n\nfunc (bw *BytesWriter) WriteAt(p []byte, offset int64) (n int, err error) {\n\t_o, err := castInt64ToInt(offset)\n\tif err != nil {\n\t\treturn -1, err\n\t}\n\treq := _o + len(p)\n\tif req > len(bw.buf) {\n\t\tbw.grow(req)\n\t\tbw.buf = bw.buf[0:req]\n\t}\n\tcopy(bw.buf[_o:req], p)\n\treturn len(p), nil\n}\n\nfunc (bw *BytesWriter) Size() int64 {\n\treturn int64(len(bw.buf))\n}\n\nfunc (bw *BytesWriter) Bytes() []byte {\n\treturn bw.buf\n}\n\nfunc IsEOF(e error) bool {\n\treturn e == io.EOF || e == io.ErrUnexpectedEOF\n}\n\nfunc IsTimeout(e error) bool {\n\tt, ok := e.(interface{ Timeout() bool })\n\tif ok {\n\t\treturn t.Timeout()\n\t}\n\treturn false\n}\n"
  },
  {
    "path": "logging.go",
    "content": "package main\n\ntype DebugLogger interface {\n\tDebug(args ...interface{})\n}\n\ntype InfoLogger interface {\n\tInfo(args ...interface{})\n}\n\ntype ErrorLogger interface {\n\tError(args ...interface{})\n}\n"
  },
  {
    "path": "main.go",
    "content": "package main\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"net\"\n\t\"os\"\n\t\"os/signal\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"golang.org/x/crypto/ssh\"\n)\n\nvar (\n\tconfigFile string\n\tbind       string\n\tdebug      bool\n)\n\nfunc init() {\n\tflag.StringVar(&configFile, \"config\", \"s3-sftp-proxy.toml\", \"configuration file\")\n\tflag.StringVar(&bind, \"bind\", \"\", \"listen on addr:port\")\n\tflag.BoolVar(&debug, \"debug\", false, \"turn on debugging output\")\n}\n\nfunc buildSSHServerConfig(buckets *S3Buckets, cfg *S3SFTPProxyConfig) (*ssh.ServerConfig, error) {\n\tpem, err := ioutil.ReadFile(cfg.HostKeyFile)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, `failed to open \"%s\"`, cfg.HostKeyFile)\n\t}\n\tkey, err := ssh.ParseRawPrivateKey(pem)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, `failed to parse host key \"%s\"`, cfg.HostKeyFile)\n\t}\n\tc := &ssh.ServerConfig{\n\t\tPasswordCallback: func(c ssh.ConnMetadata, passwd []byte) (*ssh.Permissions, error) {\n\t\t\tbucket, ok := buckets.UserToBucketMap[c.User()]\n\t\t\tif !ok {\n\t\t\t\treturn nil, fmt.Errorf(\"unknown user: %s\", c.User())\n\t\t\t}\n\t\t\tu := bucket.Users.Lookup(c.User())\n\t\t\tif u.Password != \"\" && u.Password == string(passwd) {\n\t\t\t\treturn nil, nil\n\t\t\t}\n\t\t\treturn nil, fmt.Errorf(\"passwords do not match\")\n\t\t},\n\t\tPublicKeyCallback: func(c ssh.ConnMetadata, key ssh.PublicKey) (*ssh.Permissions, error) {\n\t\t\tbucket, ok := buckets.UserToBucketMap[c.User()]\n\t\t\tif !ok {\n\t\t\t\treturn nil, fmt.Errorf(\"unknown user: %s\", c.User())\n\t\t\t}\n\t\t\tu := bucket.Users.Lookup(c.User())\n\t\t\tif u.PublicKeys != nil {\n\t\t\t\tkeyMarshaled := key.Marshal()\n\t\t\t\tfor _, herKey := range u.PublicKeys {\n\t\t\t\t\tif herKey.Type() == key.Type() && len(herKey.Marshal()) == len(keyMarshaled) && bytes.Compare(herKey.Marshal(), keyMarshaled) == 0 {\n\t\t\t\t\t\treturn &ssh.Permissions{\n\t\t\t\t\t\t\tExtensions: map[string]string{\n\t\t\t\t\t\t\t\t\"pubkey-fp\": ssh.FingerprintSHA256(key),\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}, nil\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil, fmt.Errorf(\"public keys do not match\")\n\t\t},\n\t\tKeyboardInteractiveCallback: func(c ssh.ConnMetadata, client ssh.KeyboardInteractiveChallenge) (*ssh.Permissions, error) {\n\t\t\tbucket, ok := buckets.UserToBucketMap[c.User()]\n\t\t\tif !ok {\n\t\t\t\treturn nil, fmt.Errorf(\"unknown user: %s\", c.User())\n\t\t\t}\n\t\t\tif !bucket.KeyboardInteractiveAuthEnabled {\n\t\t\t\treturn nil, fmt.Errorf(\"keyboard interactive authentication not enabled\")\n\t\t\t}\n\t\t\tu := bucket.Users.Lookup(c.User())\n\t\t\tif u.Password == \"\" {\n\t\t\t\treturn nil, fmt.Errorf(\"no credentials are present\")\n\t\t\t}\n\t\t\tanswers, err := client(u.Name, \"\", []string{\"Password: \"}, []bool{false})\n\t\t\tif err != nil {\n\t\t\t\treturn nil, errors.Wrapf(err, \"keyboard interactive conversation failed\")\n\t\t\t}\n\t\t\tif answers[0] != u.Password {\n\t\t\t\treturn nil, fmt.Errorf(\"passwords do not match\")\n\t\t\t}\n\t\t\treturn nil, nil\n\t\t},\n\t\tBannerCallback: func(c ssh.ConnMetadata) string {\n\t\t\treturn cfg.Banner\n\t\t},\n\t}\n\tsgn, err := ssh.NewSignerFromKey(key)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tc.AddHostKey(sgn)\n\treturn c, nil\n}\n\nfunc bail(msg string, status ...interface{}) {\n\tos.Stderr.Write([]byte(msg + \"\\n\"))\n\tstatusCode := 1\n\tif len(status) > 0 {\n\t\tvar ok bool\n\t\tstatusCode, ok = status[0].(int)\n\t\tif !ok {\n\t\t\tpanic(\"invalid argument for bail()\")\n\t\t}\n\t}\n\tos.Exit(statusCode)\n}\n\nfunc main() {\n\tflag.Parse()\n\tcfg, err := ReadConfigFromFile(configFile)\n\tif err != nil {\n\t\tbail(err.Error())\n\t}\n\n\tuStores, err := NewUserStoresFromConfig(cfg)\n\tif err != nil {\n\t\tbail(err.Error())\n\t}\n\n\tbuckets, err := NewS3BucketFromConfig(uStores, cfg)\n\tif err != nil {\n\t\tbail(err.Error())\n\t}\n\n\tsCfg, err := buildSSHServerConfig(buckets, cfg)\n\tif err != nil {\n\t\tbail(err.Error())\n\t}\n\n\t_bind := bind\n\tif _bind == \"\" {\n\t\t_bind = cfg.Bind\n\t\tif _bind == \"\" {\n\t\t\t_bind = \":10022\"\n\t\t}\n\t}\n\n\tlogger := logrus.New()\n\tif debug {\n\t\tlogger.SetLevel(logrus.DebugLevel)\n\t}\n\n\tlsnr, err := net.Listen(\"tcp\", _bind)\n\tif err != nil {\n\t\tbail(err.Error())\n\t}\n\tdefer lsnr.Close()\n\tlogger.Info(\"Listen on \", _bind)\n\n\tctx, cancel := context.WithCancel(context.Background())\n\tdefer cancel()\n\n\tsigChan := make(chan os.Signal)\n\tsignal.Notify(sigChan, os.Interrupt)\n\n\terrChan := make(chan error)\n\tgo func() {\n\t\terrChan <- (&Server{\n\t\t\tS3Buckets:                buckets,\n\t\t\tServerConfig:             sCfg,\n\t\t\tLog:                      logger,\n\t\t\tReaderLookbackBufferSize: *cfg.ReaderLookbackBufferSize,\n\t\t\tReaderMinChunkSize:       *cfg.ReaderMinChunkSize,\n\t\t\tListerLookbackBufferSize: *cfg.ListerLookbackBufferSize,\n\t\t\tPhantomObjectMap:         NewPhantomObjectMap(),\n\t\t\tNow:                      time.Now,\n\t\t}).RunListenerEventLoop(ctx, lsnr.(*net.TCPListener))\n\t}()\n\nouter:\n\tfor {\n\t\tselect {\n\t\tcase err = <-errChan:\n\t\t\tif err != nil {\n\t\t\t\tbail(err.Error())\n\t\t\t}\n\t\t\tbreak outer\n\t\tcase <-sigChan:\n\t\t\tcancel()\n\t\t}\n\t}\n}\n"
  },
  {
    "path": "merged_context.go",
    "content": "package main\n\nimport (\n\t\"context\"\n\t\"reflect\"\n\t\"time\"\n)\n\ntype mergedContext struct {\n\tctxs     []context.Context\n\tdoneChan chan struct{}\n\terr      error\n}\n\nfunc (ctxs *mergedContext) Deadline() (time.Time, bool) {\n\tretval := time.Time{}\n\tretvalAvail := false\n\tfor _, ctx := range ctxs.ctxs {\n\t\tif dl, ok := ctx.Deadline(); ok {\n\t\t\tif !retval.IsZero() || retval.After(dl) {\n\t\t\t\tretval = dl\n\t\t\t\tretvalAvail = true\n\t\t\t}\n\t\t}\n\t}\n\treturn retval, retvalAvail\n}\n\nfunc (ctxs *mergedContext) Done() <-chan struct{} {\n\treturn ctxs.doneChan\n}\n\nfunc (ctxs *mergedContext) Err() error {\n\treturn ctxs.err\n}\n\nfunc (ctxs *mergedContext) Value(key interface{}) interface{} {\n\tfor _, ctx := range ctxs.ctxs {\n\t\tv := ctx.Value(key)\n\t\tif v != nil {\n\t\t\treturn v\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc (ctxs *mergedContext) watcher() {\n\tif len(ctxs.ctxs) == 2 {\n\t\tgo func() {\n\t\t\tselect {\n\t\t\tcase <-ctxs.ctxs[0].Done():\n\t\t\t\tctxs.err = ctxs.ctxs[0].Err()\n\t\t\tcase <-ctxs.ctxs[1].Done():\n\t\t\t\tctxs.err = ctxs.ctxs[0].Err()\n\t\t\t}\n\t\t\tclose(ctxs.doneChan)\n\t\t}()\n\t} else {\n\t\tcases := []reflect.SelectCase{}\n\t\tfor _, ctx := range ctxs.ctxs {\n\t\t\tcases = append(cases, reflect.SelectCase{\n\t\t\t\tDir:  reflect.SelectRecv,\n\t\t\t\tChan: reflect.ValueOf(ctx.Done()),\n\t\t\t})\n\t\t}\n\t\tgo func() {\n\t\t\tchosen, _, _ := reflect.Select(cases)\n\t\t\tctxs.err = ctxs.ctxs[chosen].Err()\n\t\t\tclose(ctxs.doneChan)\n\t\t}()\n\t}\n}\n\nfunc combineContext(ctxs ...context.Context) context.Context {\n\tif len(ctxs) == 1 {\n\t\treturn ctxs[0]\n\t}\n\tctx := &mergedContext{\n\t\tctxs:     ctxs,\n\t\tdoneChan: make(chan struct{}),\n\t\terr:      nil,\n\t}\n\tctx.watcher()\n\treturn ctx\n}\n"
  },
  {
    "path": "path.go",
    "content": "package main\n\nimport \"strings\"\n\ntype Path []string\n\nfunc splitIntoPathInner(p Path, path string, state int) Path {\n\ts := 0\n\ti := 0\n\tc := 0\n\tfor c >= 0 {\n\t\tif i < len(path) {\n\t\t\tc = int(path[i])\n\t\t} else {\n\t\t\tc = -1\n\t\t}\n\t\tswitch state {\n\t\tcase 0:\n\t\t\tif c == '/' {\n\t\t\t\ti++\n\t\t\t} else {\n\t\t\t\tstate = 1\n\t\t\t\ts = i\n\t\t\t}\n\t\tcase 1:\n\t\t\tif c == '/' || c < 0 {\n\t\t\t\tp = append(p, path[s:i])\n\t\t\t\tstate = 0\n\t\t\t} else {\n\t\t\t\ti++\n\t\t\t}\n\t\t}\n\t}\n\treturn p\n}\n\nfunc SplitIntoPathAsAbs(path string) Path {\n\tif path == \"\" {\n\t\treturn Path{}\n\t}\n\treturn splitIntoPathInner(Path{\"\"}, path, 0)\n}\n\nfunc SplitIntoPath(path string) Path {\n\tif path == \"\" {\n\t\treturn Path{}\n\t}\n\treturn splitIntoPathInner(Path{}, path, 1)\n}\n\nfunc (p Path) Canonicalize() Path {\n\tretval := make(Path, 0, len(p))\n\tfor _, c := range p {\n\t\tswitch c {\n\t\tcase \".\":\n\t\t\tcontinue\n\t\tcase \"..\":\n\t\t\tif len(retval) > 0 && retval[len(retval)-1] != \"\" {\n\t\t\t\tretval = retval[:len(retval)-1]\n\t\t\t}\n\t\tdefault:\n\t\t\tretval = append(retval, c)\n\t\t}\n\t}\n\treturn retval\n}\n\nfunc (p Path) IsEmpty() bool {\n\treturn len(p) == 0\n}\n\nfunc (p Path) IsRoot() bool {\n\treturn len(p) == 1 && p[0] == \"\"\n}\n\nfunc (p Path) IsAbs() bool {\n\treturn len(p) > 0 && p[0] == \"\"\n}\n\nfunc (p Path) Join(another Path) Path {\n\tif len(another) > 0 && another[0] == \"\" {\n\t\treturn append(p, another[1:]...)\n\t} else {\n\t\treturn append(p, another...)\n\t}\n}\n\nfunc (p Path) String() string {\n\treturn strings.Join(p, \"/\")\n}\n\nfunc (p Path) IsPrefixed(another Path) bool {\n\tif len(p) < len(another) {\n\t\treturn false\n\t}\n\tfor i, c := range another {\n\t\tif p[i] != c {\n\t\t\treturn false\n\t\t}\n\t}\n\treturn true\n}\n\nfunc (p Path) Prefix() Path {\n\tif len(p) == 0 {\n\t\treturn p\n\t} else if len(p) == 1 {\n\t\tif p[0] == \"\" {\n\t\t\treturn Path{\"\"}\n\t\t} else {\n\t\t\treturn Path{}\n\t\t}\n\t} else {\n\t\treturn p[:len(p)-1]\n\t}\n}\n\nfunc (p Path) BasePart() Path {\n\tif len(p) == 0 {\n\t\treturn p\n\t} else if len(p) == 1 {\n\t\tif p[0] == \"\" {\n\t\t\treturn Path{\"\"}\n\t\t} else {\n\t\t\treturn Path{}\n\t\t}\n\t} else {\n\t\treturn p[len(p)-1:]\n\t}\n}\n\nfunc (p Path) Base() string {\n\tif len(p) == 0 {\n\t\treturn \"\"\n\t} else if len(p) == 1 {\n\t\tif p[0] == \"\" {\n\t\t\treturn \"/\"\n\t\t} else {\n\t\t\treturn \"\"\n\t\t}\n\t} else {\n\t\treturn p[len(p)-1]\n\t}\n}\n\nfunc (p Path) Equal(p2 Path) bool {\n\tif len(p) != len(p2) {\n\t\treturn false\n\t}\n\tfor i := 0; i < len(p); i++ {\n\t\tif p[i] != p2[i] {\n\t\t\treturn false\n\t\t}\n\t}\n\treturn true\n}\n"
  },
  {
    "path": "path_test.go",
    "content": "package main\n\nimport (\n\t\"github.com/stretchr/testify/assert\"\n\t\"testing\"\n)\n\nfunc TestSplitIntoPath(t *testing.T) {\n\tassert.Equal(t, Path{}, SplitIntoPath(\"\"))\n\tassert.Equal(t, Path{\"abc\"}, SplitIntoPath(\"abc\"))\n\tassert.Equal(t, Path{\"abc\", \"bcd\"}, SplitIntoPath(\"abc/bcd\"))\n\tassert.Equal(t, Path{\"abc\", \"bcd\"}, SplitIntoPath(\"abc/bcd/\"))\n\tassert.Equal(t, Path{\"abc\", \"bcd\"}, SplitIntoPath(\"abc/bcd///\"))\n\tassert.Equal(t, Path{\"abc\", \"bcd\", \"cde\"}, SplitIntoPath(\"abc/bcd//cde\"))\n\tassert.Equal(t, Path{\"\"}, SplitIntoPath(\"/\"))\n\tassert.Equal(t, Path{\"\"}, SplitIntoPath(\"//\"))\n\tassert.Equal(t, Path{\"\", \"abc\", \"bcd\"}, SplitIntoPath(\"//abc//bcd\"))\n}\n\nfunc TestSplitIntoPathAbsolute(t *testing.T) {\n\tassert.Equal(t, Path{}, SplitIntoPathAsAbs(\"\"))\n\tassert.Equal(t, Path{\"\", \"abc\"}, SplitIntoPathAsAbs(\"abc\"))\n\tassert.Equal(t, Path{\"\", \"abc\", \"bcd\"}, SplitIntoPathAsAbs(\"abc/bcd\"))\n\tassert.Equal(t, Path{\"\", \"abc\", \"bcd\"}, SplitIntoPathAsAbs(\"abc/bcd/\"))\n\tassert.Equal(t, Path{\"\", \"abc\", \"bcd\"}, SplitIntoPathAsAbs(\"abc/bcd///\"))\n\tassert.Equal(t, Path{\"\", \"abc\", \"bcd\", \"cde\"}, SplitIntoPathAsAbs(\"abc/bcd//cde\"))\n\tassert.Equal(t, Path{\"\"}, SplitIntoPathAsAbs(\"/\"))\n\tassert.Equal(t, Path{\"\"}, SplitIntoPathAsAbs(\"//\"))\n\tassert.Equal(t, Path{\"\", \"abc\", \"bcd\"}, SplitIntoPathAsAbs(\"//abc//bcd\"))\n}\n"
  },
  {
    "path": "phantom_object_map.go",
    "content": "package main\n\nimport (\n\t\"sync\"\n\t\"time\"\n)\n\ntype PhantomObjectInfo struct {\n\tKey          Path\n\tLastModified time.Time\n\tSize         int64\n\tOpaque       interface{}\n\tMtx          sync.Mutex\n}\n\nfunc (info *PhantomObjectInfo) GetOne() PhantomObjectInfo {\n\tinfo.Mtx.Lock()\n\tdefer info.Mtx.Unlock()\n\treturn *info\n}\n\nfunc (info *PhantomObjectInfo) setKey(v Path) {\n\tinfo.Mtx.Lock()\n\tdefer info.Mtx.Unlock()\n\tinfo.Key = v\n}\n\nfunc (info *PhantomObjectInfo) SetLastModified(v time.Time) {\n\tinfo.Mtx.Lock()\n\tdefer info.Mtx.Unlock()\n\tinfo.LastModified = v\n}\n\nfunc (info *PhantomObjectInfo) SetSize(v int64) {\n\tinfo.Mtx.Lock()\n\tdefer info.Mtx.Unlock()\n\tinfo.Size = v\n}\n\ntype phantomObjectInfoMap map[string]*PhantomObjectInfo\n\ntype PhantomObjectMap struct {\n\tperPrefixObjects map[string]phantomObjectInfoMap\n\tptrToPOIMMapMap  map[*PhantomObjectInfo]phantomObjectInfoMap\n\tmtx              sync.Mutex\n}\n\nfunc (pom *PhantomObjectMap) add(info *PhantomObjectInfo) bool {\n\tprefix := info.Key.Prefix().String()\n\tm := pom.perPrefixObjects[prefix]\n\tif m == nil {\n\t\tm = phantomObjectInfoMap{}\n\t\tpom.perPrefixObjects[prefix] = m\n\t}\n\tprevInfo := m[info.Key.Base()]\n\tm[info.Key.Base()] = info\n\tpom.ptrToPOIMMapMap[info] = m\n\tif prevInfo != nil {\n\t\tdelete(pom.ptrToPOIMMapMap, prevInfo)\n\t}\n\treturn prevInfo == nil\n}\n\nfunc (pom *PhantomObjectMap) Add(info *PhantomObjectInfo) bool {\n\tpom.mtx.Lock()\n\tdefer pom.mtx.Unlock()\n\treturn pom.add(info)\n}\n\nfunc (pom *PhantomObjectMap) remove(key Path) *PhantomObjectInfo {\n\tprefix := key.Prefix().String()\n\tm := pom.perPrefixObjects[prefix]\n\tif m == nil {\n\t\treturn nil\n\t}\n\tinfo := m[key.Base()]\n\tif info == nil {\n\t\treturn nil\n\t}\n\tdelete(m, key.Base())\n\tif len(m) == 0 {\n\t\tdelete(pom.perPrefixObjects, prefix)\n\t}\n\tdelete(pom.ptrToPOIMMapMap, info)\n\treturn info\n}\n\nfunc (pom *PhantomObjectMap) Remove(key Path) *PhantomObjectInfo {\n\tpom.mtx.Lock()\n\tdefer pom.mtx.Unlock()\n\treturn pom.remove(key)\n}\n\nfunc (pom *PhantomObjectMap) removeByInfoPtr(info *PhantomObjectInfo) bool {\n\tm := pom.ptrToPOIMMapMap[info]\n\tif m == nil {\n\t\treturn false\n\t}\n\tdelete(m, info.Key.Base())\n\tif len(m) == 0 {\n\t\tdelete(pom.perPrefixObjects, info.Key.Prefix().String())\n\t}\n\tdelete(pom.ptrToPOIMMapMap, info)\n\treturn true\n}\n\nfunc (pom *PhantomObjectMap) RemoveByInfoPtr(info *PhantomObjectInfo) bool {\n\tpom.mtx.Lock()\n\tdefer pom.mtx.Unlock()\n\treturn pom.removeByInfoPtr(info)\n}\n\nfunc (pom *PhantomObjectMap) rename(old, new Path) bool {\n\tinfo := pom.remove(old)\n\tif info == nil {\n\t\treturn false\n\t}\n\tinfo.setKey(new)\n\tpom.add(info)\n\treturn true\n}\n\nfunc (pom *PhantomObjectMap) Rename(old, new Path) bool {\n\tpom.mtx.Lock()\n\tdefer pom.mtx.Unlock()\n\treturn pom.rename(old, new)\n}\n\nfunc (pom *PhantomObjectMap) get(p Path) *PhantomObjectInfo {\n\tm := pom.perPrefixObjects[p.Prefix().String()]\n\tif m == nil {\n\t\treturn nil\n\t}\n\treturn m[p.Base()]\n}\n\nfunc (pom *PhantomObjectMap) Get(p Path) *PhantomObjectInfo {\n\tpom.mtx.Lock()\n\tdefer pom.mtx.Unlock()\n\treturn pom.get(p)\n}\n\nfunc (pom *PhantomObjectMap) List(p Path) []*PhantomObjectInfo {\n\tpom.mtx.Lock()\n\tdefer pom.mtx.Unlock()\n\n\tm := pom.perPrefixObjects[p.String()]\n\tretval := make([]*PhantomObjectInfo, 0, len(m))\n\tfor _, info := range m {\n\t\tretval = append(retval, info)\n\t}\n\treturn retval\n}\n\nfunc (pom *PhantomObjectMap) Size() int {\n\tpom.mtx.Lock()\n\tdefer pom.mtx.Unlock()\n\n\treturn len(pom.ptrToPOIMMapMap)\n}\n\nfunc NewPhantomObjectMap() *PhantomObjectMap {\n\treturn &PhantomObjectMap{\n\t\tperPrefixObjects: map[string]phantomObjectInfoMap{},\n\t\tptrToPOIMMapMap:  map[*PhantomObjectInfo]phantomObjectInfoMap{},\n\t}\n}\n"
  },
  {
    "path": "phantom_object_map_test.go",
    "content": "package main\n\nimport (\n\t\"github.com/stretchr/testify/assert\"\n\t\"testing\"\n)\n\nfunc TestPhantomObjectMapAdd(t *testing.T) {\n\tpom := NewPhantomObjectMap()\n\tassert.Equal(t, true, pom.Add(&PhantomObjectInfo{Key: Path{\"\", \"a\", \"b\"}}))\n\tassert.Equal(t, 1, pom.Size())\n\tassert.Equal(t, false, pom.Add(&PhantomObjectInfo{Key: Path{\"\", \"a\", \"b\"}}))\n\tassert.Equal(t, 1, pom.Size())\n\tassert.Equal(t, true, pom.Add(&PhantomObjectInfo{Key: Path{\"\", \"a\", \"c\"}}))\n\tassert.Equal(t, 2, pom.Size())\n\tassert.Equal(t, true, pom.Add(&PhantomObjectInfo{Key: Path{\"\", \"a\", \"b\", \"c\"}}))\n\tassert.Equal(t, 3, pom.Size())\n}\n\nfunc TestPhantomObjectMapRemove(t *testing.T) {\n\tpom := NewPhantomObjectMap()\n\to1 := &PhantomObjectInfo{Key: Path{\"\", \"a\", \"b\"}}\n\to2 := &PhantomObjectInfo{Key: Path{\"\", \"a\", \"b\"}}\n\to3 := &PhantomObjectInfo{Key: Path{\"\", \"a\", \"c\"}}\n\to4 := &PhantomObjectInfo{Key: Path{\"\", \"a\", \"b\", \"c\"}}\n\tassert.Equal(t, true, pom.Add(o1))\n\tassert.Equal(t, 1, pom.Size())\n\tassert.Equal(t, false, pom.Add(o2))\n\tassert.Equal(t, 1, pom.Size())\n\tassert.Equal(t, true, pom.Add(o3))\n\tassert.Equal(t, 2, pom.Size())\n\tassert.Equal(t, true, pom.Add(o4))\n\tassert.Equal(t, 3, pom.Size())\n\tassert.Equal(t, o3, pom.Remove(Path{\"\", \"a\", \"c\"}))\n\tassert.Nil(t, pom.Get(Path{\"\", \"a\", \"c\"}))\n\tassert.Equal(t, 2, pom.Size())\n\tassert.Nil(t, pom.Remove(Path{\"\", \"a\", \"c\"}))\n\tassert.Equal(t, 2, pom.Size())\n\tassert.Equal(t, o2, pom.Remove(Path{\"\", \"a\", \"b\"}))\n\tassert.Equal(t, 1, pom.Size())\n\tassert.Nil(t, pom.Get(Path{\"\", \"a\", \"b\"}))\n\n}\n"
  },
  {
    "path": "s3-sftp-proxy.example.toml",
    "content": "host_key_file = \"./host_key\"\n\n[buckets.test]\n# endpoint = \"http://endpoint\"\n# s3_force_path_style = false\n# disable_ssl = false\nbucket_url = \"s3://BUCKET/PREFIX\"\n# bucket = BUCKET\n# key_prefix = PREFIX\nprofile = \"xxx\"\nregion = \"ap-northeast-1\"\nauth = \"test\"\n\n# [buckets.test.credentials]\n# aws_access_key_id = \"aaa\"\n# aws_secret_access_key = \"bbb\"\n\n[auth.test]\ntype = \"inplace\"\n\n[auth.test.users.user01]\npassword = \"test\"\npublic_keys = \"\"\"\n...\n\"\"\"\n\n[auth.test.users.user02]\npassword = \"test\"\npublic_keys = \"\"\"\n...\n\"\"\"\n"
  },
  {
    "path": "server.go",
    "content": "package main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/pkg/sftp\"\n\t\"golang.org/x/crypto/ssh\"\n)\n\ntype Server struct {\n\t*ssh.ServerConfig\n\t*S3Buckets\n\t*PhantomObjectMap\n\tReaderLookbackBufferSize int\n\tReaderMinChunkSize       int\n\tListerLookbackBufferSize int\n\tLog                      interface {\n\t\tDebugLogger\n\t\tInfoLogger\n\t\tErrorLogger\n\t}\n\tNow func() time.Time\n}\n\nfunc asHandlers(handlers interface {\n\tsftp.FileReader\n\tsftp.FileWriter\n\tsftp.FileCmder\n\tsftp.FileLister\n}) sftp.Handlers {\n\treturn sftp.Handlers{handlers, handlers, handlers, handlers}\n}\n\nfunc (s *Server) HandleChannel(ctx context.Context, bucket *S3Bucket, sshCh ssh.Channel, reqs <-chan *ssh.Request) {\n\tdefer s.Log.Debug(\"HandleChannel ended\")\n\tserver := sftp.NewRequestServer(\n\t\tsshCh,\n\t\tasHandlers(\n\t\t\t&S3BucketIO{\n\t\t\t\tCtx:                      ctx,\n\t\t\t\tBucket:                   bucket,\n\t\t\t\tReaderLookbackBufferSize: s.ReaderLookbackBufferSize,\n\t\t\t\tReaderMinChunkSize:       s.ReaderMinChunkSize,\n\t\t\t\tListerLookbackBufferSize: s.ListerLookbackBufferSize,\n\t\t\t\tLog:                      s.Log,\n\t\t\t\tPhantomObjectMap:         s.PhantomObjectMap,\n\t\t\t\tPerms:                    bucket.Perms,\n\t\t\t\tServerSideEncryption:     &bucket.ServerSideEncryption,\n\t\t\t\tNow:                      s.Now,\n\t\t\t},\n\t\t),\n\t)\n\n\tinnerCtx, cancel := context.WithCancel(ctx)\n\tdefer cancel()\n\n\twg := sync.WaitGroup{}\n\twg.Add(1)\n\tgo func() {\n\t\tdefer s.Log.Debug(\"HandleChannel.discardRequest ended\")\n\t\tdefer wg.Done()\n\t\tdefer cancel()\n\touter:\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-innerCtx.Done():\n\t\t\t\tbreak outer\n\t\t\tcase req := <-reqs:\n\t\t\t\tif req == nil {\n\t\t\t\t\tbreak outer\n\t\t\t\t}\n\t\t\t\tok := false\n\t\t\t\tif req.Type == \"subsystem\" && string(req.Payload[4:]) == \"sftp\" {\n\t\t\t\t\tok = true\n\t\t\t\t}\n\t\t\t\treq.Reply(ok, nil)\n\t\t\t}\n\t\t}\n\t}()\n\n\twg.Add(1)\n\tgo func() {\n\t\tdefer s.Log.Debug(\"HandleChannel.serve ended\")\n\t\tdefer wg.Done()\n\t\tdefer cancel()\n\t\tgo func() {\n\t\t\t<-innerCtx.Done()\n\t\t\tserver.Close()\n\t\t}()\n\t\tif err := server.Serve(); err != io.EOF {\n\t\t\ts.Log.Error(err.Error())\n\t\t}\n\t}()\n\n\twg.Wait()\n}\n\nfunc (s *Server) HandleClient(ctx context.Context, conn *net.TCPConn) error {\n\tdefer s.Log.Debug(\"HandleClient ended\")\n\tdefer func() {\n\t\tF(s.Log.Info, \"connection from client %s closed\", conn.RemoteAddr().String())\n\t\tconn.Close()\n\t}()\n\tF(s.Log.Info, \"connected from client %s\", conn.RemoteAddr().String())\n\n\tinnerCtx, cancel := context.WithCancel(ctx)\n\tdefer cancel()\n\n\tgo func() {\n\t\t<-innerCtx.Done()\n\t\tconn.SetDeadline(time.Unix(1, 0))\n\t}()\n\n\t// Before use, a handshake must be performed on the incoming net.Conn.\n\tsconn, chans, reqs, err := ssh.NewServerConn(conn, s.ServerConfig)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tF(s.Log.Info, \"user %s logged in\", sconn.User())\n\tbucket, ok := s.UserToBucketMap[sconn.User()]\n\tif !ok {\n\t\treturn fmt.Errorf(\"unknown error: no bucket designated to user %s found\", sconn.User())\n\t}\n\n\twg := sync.WaitGroup{}\n\n\twg.Add(1)\n\tgo func(reqs <-chan *ssh.Request) {\n\t\tdefer wg.Done()\n\t\tdefer s.Log.Debug(\"HandleClient.requestHandler ended\")\n\t\tfor _ = range reqs {\n\t\t}\n\t}(reqs)\n\n\twg.Add(1)\n\tgo func(chans <-chan ssh.NewChannel) {\n\t\tdefer wg.Done()\n\t\tdefer cancel()\n\t\tdefer s.Log.Debug(\"HandleClient.channelHandler ended\")\n\t\tfor newSSHCh := range chans {\n\t\t\tif newSSHCh.ChannelType() != \"session\" {\n\t\t\t\tnewSSHCh.Reject(ssh.UnknownChannelType, \"unknown channel type\")\n\t\t\t\tF(s.Log.Info, \"unknown channel type: %s\", newSSHCh.ChannelType())\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tF(s.Log.Info, \"channel: %s\", newSSHCh.ChannelType())\n\n\t\t\tsshCh, reqs, err := newSSHCh.Accept()\n\t\t\tif err != nil {\n\t\t\t\tF(s.Log.Error, \"could not accept channel: %s\", err.Error())\n\t\t\t\tbreak\n\t\t\t}\n\n\t\t\twg.Add(1)\n\t\t\tgo func() {\n\t\t\t\tdefer wg.Done()\n\t\t\t\ts.HandleChannel(innerCtx, bucket, sshCh, reqs)\n\t\t\t}()\n\t\t}\n\t}(chans)\n\n\twg.Wait()\n\treturn nil\n}\n\nfunc (s *Server) RunListenerEventLoop(ctx context.Context, lsnr *net.TCPListener) error {\n\tdefer s.Log.Debug(\"RunListenerEventLoop ended\")\n\n\twg := sync.WaitGroup{}\n\tconnChan := make(chan *net.TCPConn)\n\tvar err error\n\n\twg.Add(1)\n\tgo func() {\n\t\tdefer s.Log.Debug(\"RunListenerEventLoop.connHandler ended\")\n\t\tdefer wg.Done()\n\t\tdefer close(connChan)\n\touter:\n\t\tfor {\n\t\t\tvar conn *net.TCPConn\n\t\t\tconn, err = lsnr.AcceptTCP()\n\t\t\tif err != nil {\n\t\t\t\treturn\n\t\t\t}\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\tconn.Close()\n\t\t\t\tbreak outer\n\t\t\tcase connChan <- conn:\n\t\t\t}\n\t\t}\n\t}()\n\nouter:\n\tfor {\n\t\tselect {\n\t\tcase conn := <-connChan:\n\t\t\twg.Add(1)\n\t\t\tgo func() {\n\t\t\t\tdefer wg.Done()\n\t\t\t\terr := s.HandleClient(ctx, conn)\n\t\t\t\tif err != nil {\n\t\t\t\t\ts.Log.Error(err.Error())\n\t\t\t\t}\n\t\t\t}()\n\t\tcase <-ctx.Done():\n\t\t\tlsnr.SetDeadline(time.Unix(1, 0))\n\t\t\tbreak outer\n\t\t}\n\t}\n\n\t// drain\n\tfor _ = range connChan {\n\t}\n\n\twg.Wait()\n\n\tif IsTimeout(err) {\n\t\terr = nil\n\t}\n\n\treturn err\n}\n"
  },
  {
    "path": "user.go",
    "content": "package main\n\nimport (\n\t\"fmt\"\n\t\"github.com/pkg/errors\"\n\t\"golang.org/x/crypto/ssh\"\n\t\"io/ioutil\"\n)\n\ntype User struct {\n\tName       string\n\tPassword   string\n\tPublicKeys []ssh.PublicKey\n}\n\ntype UserStore struct {\n\tName     string\n\tUsers    []*User\n\tusersMap map[string]*User\n}\n\ntype UserStores map[string]UserStore\n\nfunc (us *UserStore) Add(u *User) {\n\tus.Users = append(us.Users, u)\n\tus.usersMap[u.Name] = u\n}\n\nfunc (us *UserStore) Lookup(name string) *User {\n\tu, _ := us.usersMap[name]\n\treturn u\n}\n\nfunc parseAuthorizedKeys(pubKeys []ssh.PublicKey, pubKeyFileContent []byte) ([]ssh.PublicKey, error) {\n\tfor len(pubKeyFileContent) > 0 {\n\t\tvar pubKey ssh.PublicKey\n\t\tvar err error\n\t\tpubKey, _, _, pubKeyFileContent, err = ssh.ParseAuthorizedKey(pubKeyFileContent)\n\t\tif err != nil {\n\t\t\treturn pubKeys, err\n\t\t}\n\t\tpubKeys = append(pubKeys, pubKey)\n\t}\n\treturn pubKeys, nil\n}\n\nfunc buildUsersFromAuthConfigInplace(users []*User, aCfg *AuthConfig) ([]*User, error) {\n\tfor name, params := range aCfg.Users {\n\t\tvar pubKeys []ssh.PublicKey\n\t\tif params.PublicKeys != \"\" {\n\t\t\tvar err error\n\t\t\tpubKeys, err = parseAuthorizedKeys(pubKeys, []byte(params.PublicKeys))\n\t\t\tif err != nil {\n\t\t\t\treturn users, errors.Wrapf(err, `user \"%s\"`, name)\n\t\t\t}\n\t\t}\n\t\tif params.PublicKeyFile != \"\" {\n\t\t\tvar err error\n\t\t\tpubKeysFileContent, err := ioutil.ReadFile(params.PublicKeyFile)\n\t\t\tif err != nil {\n\t\t\t\treturn users, errors.Wrapf(err, `user \"%s\"`, name)\n\t\t\t}\n\t\t\tpubKeys, err = parseAuthorizedKeys(pubKeys, pubKeysFileContent)\n\t\t\tif err != nil {\n\t\t\t\treturn users, errors.Wrapf(err, `user \"%s\"`, name)\n\t\t\t}\n\t\t}\n\t\tusers = append(users, &User{\n\t\t\tName:       name,\n\t\t\tPassword:   params.Password,\n\t\t\tPublicKeys: pubKeys,\n\t\t})\n\t}\n\treturn users, nil\n}\n\nfunc buildUsersFromAuthConfig(users []*User, aCfg *AuthConfig) ([]*User, error) {\n\tswitch aCfg.Type {\n\tcase \"inplace\":\n\t\treturn buildUsersFromAuthConfigInplace(users, aCfg)\n\tdefault:\n\t\treturn users, fmt.Errorf(\"unknown auth config type: %s\", aCfg.Type)\n\t}\n}\n\nfunc NewUserStoresFromConfig(cfg *S3SFTPProxyConfig) (UserStores, error) {\n\tuStores := UserStores{}\n\tfor name, aCfg := range cfg.AuthConfigs {\n\t\tvar err error\n\t\tvar users []*User\n\t\tusers, err = buildUsersFromAuthConfig(users, aCfg)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tusersMap := map[string]*User{}\n\t\tfor _, u := range users {\n\t\t\tusersMap[u.Name] = u\n\t\t}\n\t\tuStores[name] = UserStore{Name: name, Users: users, usersMap: usersMap}\n\t}\n\treturn uStores, nil\n}\n"
  },
  {
    "path": "utils.go",
    "content": "package main\n\nimport \"fmt\"\n\ntype PrintlnLike func(...interface{})\n\nfunc F(p PrintlnLike, f string, args ...interface{}) {\n\tp(fmt.Sprintf(f, args...))\n}\n"
  }
]