Dropbox api 上传大文件

dropbox的api前段时间大概是更新了，发现了一个新的endpoint，https://www.dropbox.com/developers/reference/api#chunked-upload 可以分段上传文件，这样就能突破以前的150MB的限制，上传大文件。以前的脚本在这里。修改一下，加上

API_CHUNKED_UPLOAD_URL='https://api-content.dropbox.com/1/chunked_upload'
API_COMMIT_CHUNKED_UPLOAD_URL='https://api-content.dropbox.com/1/commit_chunked_upload/dropbox'
let max=150000000
if [ $filesize -gt $max ];then
  let chunk=15000000
  CHUNKFILE=/dev/shm/vps-upload-chunkfile
  DD="dd if=$1 bs=1MB iflag=count_bytes of=$CHUNKFILE"
  $DD count=$chunk
  upload_id=`curl -s -H "Content-Length: $chunk" -T $CHUNKFILE "$API_CHUNKED_UPLOAD_URL?oauth_consumer_key=$APP_KEY&oauth_token=$OAUTH_ACCESS_TOKEN&oauth_signature_method=PLAINTEXT&oauth_signature=$APP_SECRET%26$OAUTH_ACCESS_TOKEN_SECRET&oauth_timestamp=$TIME_STAMP&oauth_nonce=$RANDOM" | grep -oP "\"upload_id\": \"[^\"]+\"" | awk -F'"' '{print $(NF-1)}'`
  if [ -n "$upload_id" ];then
    let offset=chunk
    while [ `echo $offset+$chunk|bc` -lt $filesize ];do
      $DD count=$chunk skip=`echo $offset/1000/1000|bc`
      curl -s -H "Content-Length: $chunk" -T $CHUNKFILE "$API_CHUNKED_UPLOAD_URL?upload_id=$upload_id&offset=$offset&oauth_consumer_key=$APP_KEY&oauth_token=$OAUTH_ACCESS_TOKEN&oauth_signature_method=PLAINTEXT&oauth_signature=$APP_SECRET%26$OAUTH_ACCESS_TOKEN_SECRET&oauth_timestamp=$TIME_STAMP&oauth_nonce=$RANDOM" -o /dev/null
      let offset=offset+chunk
    done
    $DD skip=`echo $offset/1000/1000|bc`
    curl -s -H "Content-Length: `echo $filesize-$offset|bc`" -T $CHUNKFILE "$API_CHUNKED_UPLOAD_URL?upload_id=$upload_id&offset=$offset&oauth_consumer_key=$APP_KEY&oauth_token=$OAUTH_ACCESS_TOKEN&oauth_signature_method=PLAINTEXT&oauth_signature=$APP_SECRET%26$OAUTH_ACCESS_TOKEN_SECRET&oauth_timestamp=$TIME_STAMP&oauth_nonce=$RANDOM" -o /dev/null
    curl -s -X POST "$API_COMMIT_CHUNKED_UPLOAD_URL/$DIR/$filename?upload_id=$upload_id&oauth_consumer_key=$APP_KEY&oauth_token=$OAUTH_ACCESS_TOKEN&oauth_signature_method=PLAINTEXT&oauth_signature=$APP_SECRET%26$OAUTH_ACCESS_TOKEN_SECRET&oauth_timestamp=$TIME_STAMP&oauth_nonce=$RANDOM" -o /dev/null
    rm $CHUNKFILE
  else
    echo "Something goes wrong"
  fi
else
  curl -s -H "Content-Length: $filesize" -T "$1" "$API_FILES_PUT_URL/$DIR/$filename?oauth_consumer_key=$APP_KEY&oauth_token=$OAUTH_ACCESS_TOKEN&oauth_signature_method=PLAINTEXT&oauth_signature=$APP_SECRET%26$OAUTH_ACCESS_TOKEN_SECRET&oauth_timestamp=$TIME_STAMP&oauth_nonce=$RANDOM" -o /dev/null
fi

简单来说就是判断文件大小，大于150MB就分段上传，每次上传15MB。上传完第一部分以后dropbox会返回一个id，后面上传的时候要告诉dropbox上传的是这个id所属的文件，然后全部上传完以后再告诉dropbox已经上传完了。读取文件的时候是以1MB为读取的块的大小，临时文件放在/dev/shm里，如果内存有空间就是在内存里，不然就在swap里，上传完后会删除。试了一个250MB的，工作正常。