docker 镜像压缩 docker-squash

岳来 2024-01-19 原文

docker build 通过Dockerfile 制作镜像会根据命令形成一层层layer,这样造成文件重复，镜像体积比较大；为了缩小层的数量及其大小，能够像压缩git commit一样压缩layer层，docker-squash能够执行此操作。Docker squash将压缩多个镜像layer层，以便删除存储在两头步骤中的所有数据，即删除掉重复数据。本文就docker-squash 使用效果展开讨论。

一、docker-squash 安装

pip install docker-squash

docker-squash --help
usage: docker-squash [-h] [-v] [--version] [-d] [-f FROM_LAYER] [-t TAG]
                     [-m MESSAGE] [-c] [--tmp-dir TMP_DIR]
                     [--output-path OUTPUT_PATH]
                     image

Docker layer squashing tool

positional arguments:
  image                 Image to be squashed

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         Verbose output
  --version             Show version and exit
  -d, --development     Does not clean up after failure for easier debugging
  -f FROM_LAYER, --from-layer FROM_LAYER
                        Number of layers to squash or ID of the layer (or
                        image ID or image name) to squash from. In case the
                        provided value is an integer, specified number of
                        layers will be squashed. Every layer in the image will
                        be squashed if the parameter is not provided.
  -t TAG, --tag TAG     Specify the tag to be used for the new image. If not
                        specified no tag will be applied
  -m MESSAGE, --message MESSAGE
                        Specify a commit message (comment) for the new image.
  -c, --cleanup         Remove source image from Docker after squashing
  --tmp-dir TMP_DIR     Temporary directory to be created and used
  --output-path OUTPUT_PATH
                        Path where the image should be stored after squashing.
                        If not provided, image will be loaded into Docker
                        daemon

二、使用介绍

2.1、部分压缩指定层数

首先docker history查看镜像层数

docker history java:3.0
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
63a1b1edb376        4 weeks ago         /bin/sh -c #(nop)  LABEL ImageName=reg.docke…   0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  LABEL BaseRootfsLayerCoun…   0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  LABEL BaseLayerCount=25      0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  LABEL baseOS=reg.docker.a…   0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  VOLUME [/home/admin/logs]    0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  ENTRYPOINT ["/bin/sh" "-c…   0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  ENV JVM_Xms= JVM_Xmx= JVM…   0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  ENV JAVA_HOME=/opt/taobao…   0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  ENV LANG=zh_CN.utf8          0B                  
<missing>           4 weeks ago         /bin/sh -c [[ $(uname -p) != "sw_64" ]] && r…   97MB                
<missing>           4 weeks ago         /bin/sh -c yum clean --enablerepo=* all;    …   628MB               
<missing>           4 weeks ago         /bin/sh -c #(nop) COPY file:cbad11e2b6b88c38…   19.9MB              
<missing>           4 weeks ago         /bin/sh -c #(nop) COPY file:fd3bca1e9167d639…   21MB                
<missing>           4 weeks ago         /bin/sh -c #(nop) COPY dir:2761dc242787e3ab8…   39.2MB              
<missing>           4 weeks ago         /bin/sh -c #(nop) COPY dir:8a9faa49606db326b…   8.49MB              
<missing>           4 weeks ago         /bin/sh -c #(nop)  LABEL ImageName=reg.docke…   0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  LABEL BaseRootfsLayerCoun…   0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  LABEL BaseLayerCount=10      0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  LABEL baseOS=reg.docker.a…   0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  ENTRYPOINT [""]              0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  CMD ["/sbin/init"]           0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop) COPY dir:61a850de6501dac7b…   3.03kB              
<missing>           4 weeks ago         /bin/sh -c sed -i -e 's/root:[^:]*:/root:*:/…   94.6MB              
<missing>           4 weeks ago         /bin/sh -c #(nop) COPY dir:ca4e72cff3459ccab…   395B                
<missing>           16 months ago                                                       282MB               Imported from -

使用dive 工具查看镜像效率参数

CI=true dive java:3.0
  Using default CI config
Image Source: docker://java:3.0
Fetching image... (this can take a while for large images)
Analyzing image...
  efficiency: 78.5919 %
  UserSizeByes: 90775433100 bytes (908 MB)
  SizeByes: 118928447300 bytes (1.2 GB)
  wastedBytes: 381696817 bytes (382 MB)
  userWastedPercent: 42.0485 %
Inefficient Files:
Count  Wasted Space  File Path
    2        119 MB  /usr/lib/locale/locale-archive
    ...
    2           0 B  /usr/lib/systemd/system/runlevel0.target
Results:
  FAIL: highestUserWastedPercent: too many bytes wasted, relative to the user bytes added (%-user-wasted-bytes=0.4204847104166557 > threshold=0.1)
  SKIP: highestWastedBytes: rule disabled
  FAIL: lowestEfficiency: image efficiency is too low (efficiency=0.7859187397531037 < threshold=0.9)
Result:FAIL [Total:3] [Passed:0] [Failed:2] [Warn:0] [Skipped:1

命令格式

docker-squash -f <层数，依据docker history> -t 新镜像名称 原镜像名称

如下：

docker-squash -f 11 -t java-squash-11:1.0 java:3.0
2023-01-20 13:41:43,978 root         INFO     docker-squash version 1.0.10, Docker 5138787, API 1.39...
2023-01-20 13:41:43,978 root         INFO     Using v2 image format
2023-01-20 13:41:43,982 root         INFO     Old image has 25 layers
2023-01-20 13:41:43,982 root         INFO     Checking if squashing is necessary...
2023-01-20 13:41:43,982 root         INFO     Attempting to squash last 11 layers...
2023-01-20 13:41:43,982 root         INFO     Saving image sha256:63a1b1edb376fb12838dbbef7749effc1e8e6a973d33a47aee194eff6a173edd to /tmp/docker-squash-OlTnlF/old directory...
2023-01-20 13:42:03,134 root         INFO     Image saved!
2023-01-20 13:42:03,135 root         INFO     Squashing image 'java:3.0'...
2023-01-20 13:42:03,136 root         INFO     Starting squashing...
2023-01-20 13:42:03,870 root         INFO     Squashing file '/tmp/docker-squash-OlTnlF/old/ddf4a6e4ea15a7ca3825c70e116e40f3a01f25b7df008d4517c826ef78c03dc8/layer.tar'...
2023-01-20 13:42:06,729 root         INFO     Squashing file '/tmp/docker-squash-OlTnlF/old/7be6dcb4742458db7e2c77fb1ad9fd66555e6f08d28c74592a9b973740356919/layer.tar'...
2023-01-20 13:42:19,510 root         INFO     Squashing finished!
2023-01-20 13:42:22,736 root         INFO     Original image size: 1166.43 MB
2023-01-20 13:42:22,736 root         INFO     Squashed image size: 1153.64 MB
2023-01-20 13:42:22,736 root         INFO     Image size decreased by 1.10 %
2023-01-20 13:42:22,736 root         INFO     New squashed image ID is 794121cd589230285a51d969aac97c5e936d732b0feefe5c05e78b027e850312
2023-01-20 13:42:30,244 root         INFO     Image registered in Docker daemon as java-squash-11:1.0
2023-01-20 13:42:30,369 root         INFO     Done

压缩后体积如下：

docker images | grep java
java-squash-11       1.0            794121cd5892        About a minute ago   1.17GB
java                 3.0            63a1b1edb376        4 weeks ago          1.19GB

再次查看history

docker history java-squash-11:1.0
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
794121cd5892        5 minutes ago                                                       703MB               
<missing>           4 weeks ago         /bin/sh -c #(nop) COPY file:cbad11e2b6b88c38…   19.9MB              
<missing>           4 weeks ago         /bin/sh -c #(nop) COPY file:fd3bca1e9167d639…   21MB                
<missing>           4 weeks ago         /bin/sh -c #(nop) COPY dir:2761dc242787e3ab8…   39.2MB              
<missing>           4 weeks ago         /bin/sh -c #(nop) COPY dir:8a9faa49606db326b…   8.49MB              
<missing>           4 weeks ago         /bin/sh -c #(nop)  LABEL ImageName=reg.docke…   0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  LABEL BaseRootfsLayerCoun…   0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  LABEL BaseLayerCount=10      0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  LABEL baseOS=reg.docker.a…   0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  ENTRYPOINT [""]              0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  CMD ["/sbin/init"]           0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop) COPY dir:61a850de6501dac7b…   3.03kB              
<missing>           4 weeks ago         /bin/sh -c sed -i -e 's/root:[^:]*:/root:*:/…   94.6MB              
<missing>           4 weeks ago         /bin/sh -c #(nop) COPY dir:ca4e72cff3459ccab…   395B                
<missing>           16 months ago                                                       282MB               Imported from -

再次dive 分析,镜像效率参数有些许改观

CI=true dive java-squash-11:1.0
  Using default CI config
Image Source: docker://java-squash-11:1.0
Fetching image... (this can take a while for large images)
Analyzing image...
  efficiency: 79.9717 %
  UserSizeByes: 88655803400 bytes (887 MB)
  SizeByes: 116808817600 bytes (1.2 GB)
  wastedBytes: 360500520 bytes (360 MB)
  userWastedPercent: 40.6629 %
Inefficient Files:
Count  Wasted Space  File Path
    2        119 MB  /usr/lib/locale/locale-archive
    ...
    2           0 B  /usr/lib/systemd/system/sysinit.target.wants/cryptsetup.target
Results:
  FAIL: highestUserWastedPercent: too many bytes wasted, relative to the user bytes added (%-user-wasted-bytes=0.40662935326803434 > threshold=0.1)
  SKIP: highestWastedBytes: rule disabled
  FAIL: lowestEfficiency: image efficiency is too low (efficiency=0.7997167019774828 < threshold=0.9)
Result:FAIL [Total:3] [Passed:0] [Failed:2] [Warn:0] [Skipped:1]

2.2、完全压缩

即压缩成一层

docker-squash -f 25  -t java-squash-all:1.0 java:3.0
2023-01-20 13:53:47,397 root         INFO     docker-squash version 1.0.10, Docker 5138787, API 1.39...
2023-01-20 13:53:47,397 root         INFO     Using v2 image format
2023-01-20 13:53:47,401 root         INFO     Old image has 25 layers
2023-01-20 13:53:47,401 root         INFO     Checking if squashing is necessary...
2023-01-20 13:53:47,401 root         INFO     Attempting to squash last 25 layers...
2023-01-20 13:53:47,401 root         INFO     Saving image sha256:63a1b1edb376fb12838dbbef7749effc1e8e6a973d33a47aee194eff6a173edd to /tmp/docker-squash-23fgk8/old directory...
2023-01-20 13:54:06,767 root         INFO     Image saved!
2023-01-20 13:54:06,767 root         INFO     Squashing image 'java:3.0'...
2023-01-20 13:54:06,768 root         INFO     Starting squashing...
2023-01-20 13:54:06,768 root         INFO     Squashing file '/tmp/docker-squash-23fgk8/old/ddf4a6e4ea15a7ca3825c70e116e40f3a01f25b7df008d4517c826ef78c03dc8/layer.tar'...
2023-01-20 13:54:09,616 root         INFO     Squashing file '/tmp/docker-squash-23fgk8/old/7be6dcb4742458db7e2c77fb1ad9fd66555e6f08d28c74592a9b973740356919/layer.tar'...
2023-01-20 13:54:20,751 root         INFO     Squashing file '/tmp/docker-squash-23fgk8/old/57be802a5ecce1545d1fef09e3f0b04cef12386e3995b8f98b827c1fa690f2dc/layer.tar'...
2023-01-20 13:54:20,778 root         INFO     Squashing file '/tmp/docker-squash-23fgk8/old/e113fadd42d1969d71a6d30cf402938436e9aba0de9af3ed2cdacec9e1842103/layer.tar'...
2023-01-20 13:54:20,799 root         INFO     Squashing file '/tmp/docker-squash-23fgk8/old/eca237f8359a3c25227be27fd27ebff05b510f06b9768639e3484741d9995073/layer.tar'...
2023-01-20 13:54:20,809 root         INFO     Squashing file '/tmp/docker-squash-23fgk8/old/2a47dbd3a252894e057a55d676f92625d24bfc8ad10a28b008e8b87638480ee6/layer.tar'...
2023-01-20 13:54:20,871 root         INFO     Squashing file '/tmp/docker-squash-23fgk8/old/bb207541a62d1cf5aa5dc13e971301949a22d2c3016e317d2190f1b51bde0167/layer.tar'...
2023-01-20 13:54:20,876 root         INFO     Squashing file '/tmp/docker-squash-23fgk8/old/354eb0981c6f10dcf5744386501d61c670f5bfbf77063c778b26e5c0450543f3/layer.tar'...
2023-01-20 13:54:22,489 root         INFO     Squashing file '/tmp/docker-squash-23fgk8/old/e31074992543d56a24b7da3457af34c7df7439ced6d9c0bf72ce4122fc380d81/layer.tar'...
2023-01-20 13:54:22,492 root         INFO     Squashing file '/tmp/docker-squash-23fgk8/old/1641651324846664fa3c233e4459767551e4ab9f081a718085216d30687e0877/layer.tar'...
2023-01-20 13:54:52,084 root         INFO     Squashing finished!
2023-01-20 13:54:54,922 root         INFO     Original image size: 1166.43 MB
2023-01-20 13:54:54,922 root         INFO     Squashed image size: 981.28 MB
2023-01-20 13:54:54,922 root         INFO     Image size decreased by 15.87 %
2023-01-20 13:54:54,923 root         INFO     New squashed image ID is f508f38322ae8950e49bd8cbcf140356dd236654ecd6bf303440d93081f6f7c9
2023-01-20 13:55:04,532 root         INFO     Image registered in Docker daemon as java-squash-all:1.0
2023-01-20 13:55:04,624 root         INFO     Done

docker history 查看

docker history java-squash-all:1.0
IMAGE          CREATED              CREATED BY          SIZE                COMMENT
f508f38322ae   About a minute ago                       989MB

docker images 比较一下大小

docker images | grep java
java-squash-all       1.0                 f508f38322ae        30 minutes ago      989MB
java-squash-11        1.0                 794121cd5892        42 minutes ago      1.17GB
java                  3.0                 63a1b1edb376        4 weeks ago         1.19GB

dive 分析

CI=true dive java-squash-all:1.0
  Using default CI config
Image Source: docker://java-squash-all:1.0
Fetching image... (this can take a while for large images)
Analyzing image...
  efficiency: 100.0000 %
  UserSizeByes: 0 bytes (0 B)
  SizeByes: 98904607400 bytes (989 MB)
  wastedBytes: 0 bytes (0 B)
  userWastedPercent: NaN %
Inefficient Files:
Count  Wasted Space  File Path
None
Results:
  PASS: highestUserWastedPercent
  SKIP: highestWastedBytes: rule disabled
  PASS: lowestEfficiency
Result:PASS [Total:3] [Passed:2] [Failed:0] [Warn:0] [Skipped:1]

三、总结

1、docker-squash 根据docker history 层数对镜像进行压缩
2、-f 指定的层数越大，压缩效率越好
3、结合dive 工具，更直观展示压缩效率
4、压缩后的图像可以加载回Docker守护进程，即可以通过docker images 查看
5、使用docker-squash 也存在缺点，压缩为一层时无法根据docker history查看镜像构建历史。

参考文档

1、https://lequ7.com/guan-yu-docker-chuang-jian-zui-xiao-hua-de-rong-qi-jing-xiang-er.html
2、https://mp.weixin.qq.com/s/S1Ib08SpQbf1SCbCutUoqQ
3、https://blog.csdn.net/j_starry/article/details/125185506
4、https://github.com/jwilder/docker-squash
5、https://bbs.huaweicloud.com/blogs/336684
6、https://github.com/dive/
7、https://www.cnpython.com/pypi/docker-squash
8、https://cloud.tencent.com/developer/article/1444954

有关docker 镜像压缩 docker-squash的更多相关文章

ruby - 使用 RubyZip 生成 ZIP 文件时设置压缩级别 - 2
我有一个Ruby程序，它使用rubyzip压缩XML文件的目录树。gem。我的问题是文件开始变得很重，我想提高压缩级别，因为压缩时间不是问题。我在rubyzipdocumentation中找不到一种为创建的ZIP文件指定压缩级别的方法。有人知道如何更改此设置吗？是否有另一个允许指定压缩级别的Ruby库？最佳答案这是我通过查看rubyzip内部创建的代码。level=Zlib::BEST_COMPRESSIONZip::ZipOutputStream.open(zip_file)do|zip|Dir.glob("**/*")d
Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting - 2
1.错误信息：Errorresponsefromdaemon:Gethttps://registry-1.docker.io/v2/:net/http:requestcanceledwhilewaitingforconnection(Client.Timeoutexceededwhileawaitingheaders)或者：Errorresponsefromdaemon:Gethttps://registry-1.docker.io/v2/:net/http:TLShandshaketimeout2.报错原因：docker使用的镜像网址默认为国外，下载容易超时，需要修改成国内镜像地址（首先阿里
ruby - Ruby 的压缩库？ - 2
是否有任何可用于Ruby的开源压缩/解压库？有没有人实现过LZW？或者，是否有任何使用压缩组件的开源库可以提取出来独立使用？编辑——感谢您的回答!我应该提到我必须压缩的是只驻留在数据库中的长字符串(我不会压缩文件)。此外，如果可以执行此操作的任何库都具有用于客户端压缩/分解的等效JavaScript实现，那将是理想的，因为这将用于Web应用程序。最佳答案您会在rubystdlib下找到所有已交付的ruby库的一个很好的列表.我会使用zlib库，它是开放的，无处不在，您会发现几乎所有语言的库!
ruby - 读取 zip 存档中的文件，无需解压缩存档 - 2
我有一个包含100多个zip文件的目录，我需要读取zip文件中的文件以进行一些数据处理，而无需解压缩存档。是否有一个Ruby库可以在不解压缩文件的情况下读取zip存档中的文件内容？使用rubyzip报错:require'zip'Zip::File.open('my_zip.zip')do|zip_file|#Handleentriesonebyonezip_file.eachdo|entry|#Extracttofile/directory/symlinkputs"Extracting#{entry.name}"entry.extract('here')#Readintomemoryc
ruby-on-rails - 私有(private) gem 没有安装在 docker 中 - 2
我正在尝试使用docker运行一个Rails应用程序。通过github的sshurl安装的gem很少，如下所示:Gemfilegem'swagger-docs',:git=>'git@github.com:xyz/swagger-docs.git',:branch=>'my_branch'我在docker中添加了keys，它能够克隆所需的repo并从git安装gem。DockerfileRUNmkdir-p/root/.sshCOPY./id_rsa/root/.ssh/id_rsaRUNchmod700/root/.ssh/id_rsaRUNssh-keygen-f/root/.ss
ruby - 在 Heroku Cedar 上的 Rails 3.2 中，是否有一种标准的方式来提供预压缩的 Assets ？ - 2
我有一个正在HerokuCedar堆栈上部署的Rails3.2应用程序。这意味着应用程序本身负责为其静态Assets提供服务。我希望对这些Assets进行gzip压缩，所以我在production.rb的中间件堆栈中插入了Rack::Deflater:middleware.insert_after('Rack::Cache',Rack::Deflater)...curl告诉我这与宣传的一样有效。但是，由于Heroku将全力运行rakeassets:precompile，生成一堆预gzipAssets，我很想使用它们(而不是让Rack::Deflater再次完成所有工作)。我已经看到使用
Rubyzip 与 native 操作系统压缩 - 2
我想知道与使用native操作系统库执行压缩相比，使用rubyzip压缩数据时的性能差异是什么。我正在从URL获取要压缩的数据，然后使用ZipOutputStream创建zip文件。对于native操作系统实用程序，我正在考虑使用zip工具。很高兴听到这两种方法的优缺点。最佳答案事实证明，无论是运算时间还是CPU使用率，都没有太大差异。但是在内存使用方面存在显着差异。与使用ziputil相比，使用rubyzip的过程最终会使用更多的内存。在我们的用例中，内存使用是一个重要问题，因此我们最终使用了zip实用程序。
ruby - 如何使用 rubyzip 解压缩压缩文件夹 - 2
我知道如何使用rubyzip检索普通zip文件的内容。但是我在解压缩压缩文件夹的内容时遇到了问题，我希望你们中的任何人都能帮助我。这是我用来解压的代码:Zip::ZipFile::open(@file_location)do|zip|zip.eachdo|entry|nextifentry.name=~/__MACOSX/orentry.name=~/\.DS_Store/or!entry.file?logger.debug"#{entry.name}"@data=File.new("#{Rails.root.to_s}/tmp/#{entry.name}")endendentry
ruby-on-rails - 将 Heroku 环境变量传输到 Docker 实例 - 2
我在Heroku上构建了一个必须在Docker容器内运行的RoR应用程序。为此，我使用officialDockerfile.因为它在Heroku中很常见，所以我需要一些附加组件才能使这个应用程序完全运行。在生产中，变量DATABASE_URL在我的应用程序中可用。但是，如果我尝试其他一些使用环境变量(在我的例子中是Mailtrap)的加载项，变量不会在运行时复制到实例中。所以我的问题很简单:如何让docker实例在Heroku上执行时知道环境变量？您可能会问，我已经知道我们可以在docker-compose.yml中指定一个environment指令。我想避免这种情况，以便能够通过项目
ruby - 更新 gem 时 Docker 包安装缓存问题 - 2
我在开发和生产中都使用docker，真正困扰我的一件事是docker缓存的简单性。我的ruby应用程序需要bundleinstall来安装依赖项，因此我从以下Dockerfile开始:添加GemfileGemfile添加Gemfile.lockGemfile.lock运行bundleinstall--path/root/bundle所有依赖项都被缓存，并且在我添加新gem之前效果很好。即使我添加的gem只有0.5MB，从头开始安装所有应用程序gem仍然需要10-15分钟。由于依赖项文件夹的大小(大约300MB)，然后再花10分钟来部署它。我在node_modules和npm上遇到了