From patchwork Tue Sep 3 08:57:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Trimarchi X-Patchwork-Id: 3360 Return-Path: X-Original-To: linux-amarula@patchwork.amarulasolutions.com Delivered-To: linux-amarula@patchwork.amarulasolutions.com Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by ganimede.amarulasolutions.com (Postfix) with ESMTPS id 935D93F1F3 for ; Tue, 3 Sep 2024 10:57:52 +0200 (CEST) Received: by mail-ed1-f71.google.com with SMTP id 4fb4d7f45d1cf-5c260907ecbsf683347a12.2 for ; Tue, 03 Sep 2024 01:57:52 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1725353872; cv=pass; d=google.com; s=arc-20240605; b=Gvu5V/YilncCDhpgeq3ppNSag61O47rGoclo96nqPG1rGbmWrTHE/WC4CWbT24b/Yy fv1WkyvQyczqDjN3eFcf891FERHjXCpqWPDkNNS3v7bTzqkM733ATKWRkx5JhrsAbTbh Qx6A6ZzGrtS38o1NieeYmf17r6JI/l4x1k/00pOBSxAMp6IPV/ZzXiPgwD1ilRT4cuT/ jq7f01LaQjARoYZzhkKbIPg5LN4We5vlV4mWSXO3FsEvkr8TAb7EO5lxIF1tyi8OggsY V9HDf7kXjP/QRcb60COxd/F6V4qJmkoD+YmwosxLczhSRY+eJinh7ozA4LWL/xyCYFqD Pczg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-archive:list-help:list-post:list-id :mailing-list:precedence:mime-version:message-id:date:subject:cc:to :from:dkim-signature; bh=lzWjY0TH9hflGSMg7UGhKSz8LzpXyLI/jfZ1CK+jMn4=; fh=1N+vhhayppeP1piSZ+Zv6Yxgg2pSYuW3xUVkq6UoJNw=; b=K+inVPXIWD59L1sqc79pN45CoCDH/1N2dKolqi9s7NRUMNfTSTZbfPdNtJZrzhCfbN 74yXcoseuEXL2ojueTTHJV7TBlFqYt+O/UqFPEy4Zo93kqBevAcja2Gqkfs5BkvSE0Pr +8lKnImIEKRILdBwI5hoDXf+e7kb/AoPaS8m8/nl6DSj2TKNBSK8EiBaA2KpbAQRWAA9 48xc658xHvIqrogtfrMc3/vefTM7zElF1jHqnsKiHhAOEkeoZJmsBy2ujCYwRI4+rpot 0DeIybj4x3VLZdBnQNznkYwC+qBbdOREfE9GJT6ABwWlfL/4DIHOA6xRmJKyBlz26yEt oM2g==; darn=patchwork.amarulasolutions.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amarulasolutions.com header.s=google header.b=YHvhtxTV; spf=pass (google.com: domain of michael@amarulasolutions.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=michael@amarulasolutions.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=amarulasolutions.com; dara=pass header.i=@amarulasolutions.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amarulasolutions.com; s=google; t=1725353872; x=1725958672; darn=patchwork.amarulasolutions.com; h=list-unsubscribe:list-archive:list-help:list-post:list-id :mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:message-id:date:subject:cc:to:from :from:to:cc:subject:date:message-id:reply-to; bh=lzWjY0TH9hflGSMg7UGhKSz8LzpXyLI/jfZ1CK+jMn4=; b=JeigTEOr7SMqgnMuYOWPYNHpar2Z2tt34/E78fNC7zKGOjNu+TSuqcE9TS2prZHTx1 QhAQZoLbA7QVj+l65cOS3AuR39fgaAdVzqylYLjFRSVXZYPYGGpahF7Xnahs+jw7KGiO rPjAAc5s6TLCIXRusYY0ZYB6AUvMB8jn/JWwY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725353872; x=1725958672; h=list-unsubscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :message-id:date:subject:cc:to:from:x-beenthere:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=lzWjY0TH9hflGSMg7UGhKSz8LzpXyLI/jfZ1CK+jMn4=; b=bAxj+9V7xq1A3zq2OyoIBKZy8bqH7lWAPjfMrxQDOdZi+hks8RZgae3/Yr4pDOsWHr Dy2WQ6Tb3W//EXCXFKks9GDAjAMYwQyAt8nlTQ8t0U0uto8Ub1EM4JbeT7AblqCt0yBY fkFblvX9QpGIqWsvPfSRcZCE90C3hLLsIdjGTPxZYX23BJ/4jTeMUanm/doi+QozfkRk zOgnqyl/+lJp9B92jgYK4cLuoJKqmIUOMfSw9RauNto6XGTmOUtVYiPDDkxTpECWscln NWFa8YEjo11DvVFphgO5OnckND3lBaF9QHD7EtEN/pOnAfsA+WcdsRyayiVk7JLXTC9a a8xQ== X-Forwarded-Encrypted: i=2; AJvYcCVVlBIMrS/s2frtEOTj5ak68yi69gaDbu+3twJybkrU8aB5sY3ClF5hoOHMLmVY6L7hdPDfYqnYTCsdH2F8@patchwork.amarulasolutions.com X-Gm-Message-State: AOJu0Ywd2ACXqAcnw+A83InnEgd7Ge5KVoO6HzOXl7t35dhACC+cyTdb EXSCd7NUzv7gRrFCR4oaYRsz4pOfMssTv7aD+AiZsgxjq9LWhR7dmOlu2WcFAQitkg== X-Google-Smtp-Source: AGHT+IHpEl8i2YUbNb2SXyLprS8w6EmS4fWzmELhYaCFDmLUbDTL/KjeidQu/jTn5N7BrZryVGbdHQ== X-Received: by 2002:a05:6402:2353:b0:5c0:8f7c:ad9e with SMTP id 4fb4d7f45d1cf-5c21ed3e3e3mr16360603a12.15.1725353871740; Tue, 03 Sep 2024 01:57:51 -0700 (PDT) X-BeenThere: linux-amarula@amarulasolutions.com Received: by 2002:a05:6402:4301:b0:5be:ef89:e7ab with SMTP id 4fb4d7f45d1cf-5c21ec51831ls986923a12.2.-pod-prod-05-eu; Tue, 03 Sep 2024 01:57:50 -0700 (PDT) X-Received: by 2002:a05:6402:518b:b0:5c2:6d13:c583 with SMTP id 4fb4d7f45d1cf-5c26d13c80fmr1359225a12.28.1725353869753; Tue, 03 Sep 2024 01:57:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725353869; cv=none; d=google.com; s=arc-20160816; b=zt8t9viFkTLr/MNOT8Mu2ZX9FTGI67IcotIqkWAP5MvtOP2TQEw7lOdVsrdK7r0GcX A68fSyr6bTgWW2dr6slE6y4E5M34vCVNzzn1p7OaZYx70XGhQ16Kf9pINVcAmJYoCpqt WbhvcsgiX/CIGJO/8iQBMA4LR7yVsZHeyosw3IAT7dJyrT8YsQe/XYfaHXex/3dNA3I4 n/lfNEQH33Rgkl7pnTKncKD2CZNA8fGtUquWK+sSFB/vzahAkjVzTiibAM5FS3IkdkQv +mY+K7YOJAN/+Z7zTGd1O+1wY1puRbflAjoE0RyRGaqDJxNXq7mpJpDxL2AuUiftv7Ho TgAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dkim-signature; bh=teRRsZfFE9whINoPHQ2NNCQOMz3QDap3bmLVQ/9UIjU=; fh=tFw5bWiU1jBH/a1ncq8wme8lpRu4N+XMJNVYG1SrYIo=; b=qvpgKBIHsMcyrPQV7UOm6Hd2FrOz+rYvnF/Zseke4yGH0r9+tkDPOWdXD7Pz4knrpy xcMR01GLbGwtoz7hmsPTBdA31q/PVuCXxUkF0DmMD3um5rLrLXsTOj28YPqiWoa2xg0l QnLCvcyWhpfeo8SRFuAGMkItKTlY57gC0jkwDXqqX4LAzJTmhhjtutNcT1e3N0flrecJ IUSvN1E/Teauopn/kaz17vbQJr4CGj++CsHVFiqQ6/6m+ftQA3saFYlbgbhhhRkZb9fD qQvB2UkowkKhYmvXqzYFpsqmB8IqaKFtIRZnlJOrfiVAbPQzKDaByGLXmkBMfDcs6eHc hs8Q==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@amarulasolutions.com header.s=google header.b=YHvhtxTV; spf=pass (google.com: domain of michael@amarulasolutions.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=michael@amarulasolutions.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=amarulasolutions.com; dara=pass header.i=@amarulasolutions.com Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id 4fb4d7f45d1cf-5c26f1d9a8fsor83836a12.2.2024.09.03.01.57.49 for (Google Transport Security); Tue, 03 Sep 2024 01:57:49 -0700 (PDT) Received-SPF: pass (google.com: domain of michael@amarulasolutions.com designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; X-Received: by 2002:a05:6402:1d55:b0:5c2:6311:c9d1 with SMTP id 4fb4d7f45d1cf-5c26311cb25mr3683446a12.22.1725353868659; Tue, 03 Sep 2024 01:57:48 -0700 (PDT) Received: from panicking.amarulasolutions.com (93-35-133-65.ip55.fastwebnet.it. [93.35.133.65]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c226ccfeacsm6204487a12.78.2024.09.03.01.57.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Sep 2024 01:57:48 -0700 (PDT) From: Michael Trimarchi To: buildroot@buildroot.org Cc: linux-amarula@amarulasolutions.com, Michael Trimarchi Subject: [PATCH V2 1/2] scripts/cve: Avoid to do a complete clone of cve git repository Date: Tue, 3 Sep 2024 10:57:44 +0200 Message-ID: <20240903085745.2594893-1-michael@amarulasolutions.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-Original-Sender: michael@amarulasolutions.com X-Original-Authentication-Results: mx.google.com; dkim=pass header.i=@amarulasolutions.com header.s=google header.b=YHvhtxTV; spf=pass (google.com: domain of michael@amarulasolutions.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=michael@amarulasolutions.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=amarulasolutions.com; dara=pass header.i=@amarulasolutions.com Content-Type: text/plain; charset="UTF-8" Precedence: list Mailing-list: list linux-amarula@amarulasolutions.com; contact linux-amarula+owners@amarulasolutions.com List-ID: X-Spam-Checked-In-Group: linux-amarula@amarulasolutions.com X-Google-Group-Id: 476853432473 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , Just a simple clone and pull with --depth 1 should be enough to parse the cve and generate the pkg-stats report. From a full clone and a depth-1 clone, and the size delta is 2.9GiB vs. 2.2GiB. The download size does change: from 983.55MiB down to 270.78MiB. it's a net time win too: 2m17s vs 1min7s (on a 100Mbps link). Signed-off-by: Michael Trimarchi --- V1->V2: - Add statistics from Yann E. Morin - Use git pull --depth 1 for update the repo --- support/scripts/cve.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/support/scripts/cve.py b/support/scripts/cve.py index e25825581e..dcb3a63925 100755 --- a/support/scripts/cve.py +++ b/support/scripts/cve.py @@ -72,7 +72,7 @@ class CVE: print(f"Updating from {NVD_BASE_URL}") if os.path.exists(nvd_git_dir): subprocess.check_call( - ["git", "pull"], + ["git", "pull", "--depth", "1"], cwd=nvd_git_dir, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, @@ -82,7 +82,7 @@ class CVE: # happily clones into an empty directory. os.makedirs(nvd_git_dir) subprocess.check_call( - ["git", "clone", NVD_BASE_URL, nvd_git_dir], + ["git", "clone", "--depth", "1", NVD_BASE_URL, nvd_git_dir], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, )