From patchwork Tue Sep 3 08:57:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Trimarchi X-Patchwork-Id: 3360 Return-Path: X-Original-To: linux-amarula@patchwork.amarulasolutions.com Delivered-To: linux-amarula@patchwork.amarulasolutions.com Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by ganimede.amarulasolutions.com (Postfix) with ESMTPS id 935D93F1F3 for ; Tue, 3 Sep 2024 10:57:52 +0200 (CEST) Received: by mail-ed1-f71.google.com with SMTP id 4fb4d7f45d1cf-5c260907ecbsf683347a12.2 for ; Tue, 03 Sep 2024 01:57:52 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1725353872; cv=pass; d=google.com; s=arc-20240605; b=Gvu5V/YilncCDhpgeq3ppNSag61O47rGoclo96nqPG1rGbmWrTHE/WC4CWbT24b/Yy fv1WkyvQyczqDjN3eFcf891FERHjXCpqWPDkNNS3v7bTzqkM733ATKWRkx5JhrsAbTbh Qx6A6ZzGrtS38o1NieeYmf17r6JI/l4x1k/00pOBSxAMp6IPV/ZzXiPgwD1ilRT4cuT/ jq7f01LaQjARoYZzhkKbIPg5LN4We5vlV4mWSXO3FsEvkr8TAb7EO5lxIF1tyi8OggsY V9HDf7kXjP/QRcb60COxd/F6V4qJmkoD+YmwosxLczhSRY+eJinh7ozA4LWL/xyCYFqD Pczg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-archive:list-help:list-post:list-id :mailing-list:precedence:mime-version:message-id:date:subject:cc:to :from:dkim-signature; bh=lzWjY0TH9hflGSMg7UGhKSz8LzpXyLI/jfZ1CK+jMn4=; fh=1N+vhhayppeP1piSZ+Zv6Yxgg2pSYuW3xUVkq6UoJNw=; b=K+inVPXIWD59L1sqc79pN45CoCDH/1N2dKolqi9s7NRUMNfTSTZbfPdNtJZrzhCfbN 74yXcoseuEXL2ojueTTHJV7TBlFqYt+O/UqFPEy4Zo93kqBevAcja2Gqkfs5BkvSE0Pr +8lKnImIEKRILdBwI5hoDXf+e7kb/AoPaS8m8/nl6DSj2TKNBSK8EiBaA2KpbAQRWAA9 48xc658xHvIqrogtfrMc3/vefTM7zElF1jHqnsKiHhAOEkeoZJmsBy2ujCYwRI4+rpot 0DeIybj4x3VLZdBnQNznkYwC+qBbdOREfE9GJT6ABwWlfL/4DIHOA6xRmJKyBlz26yEt oM2g==; darn=patchwork.amarulasolutions.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amarulasolutions.com header.s=google header.b=YHvhtxTV; spf=pass (google.com: domain of michael@amarulasolutions.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=michael@amarulasolutions.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=amarulasolutions.com; dara=pass header.i=@amarulasolutions.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amarulasolutions.com; s=google; t=1725353872; x=1725958672; darn=patchwork.amarulasolutions.com; h=list-unsubscribe:list-archive:list-help:list-post:list-id :mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:message-id:date:subject:cc:to:from :from:to:cc:subject:date:message-id:reply-to; bh=lzWjY0TH9hflGSMg7UGhKSz8LzpXyLI/jfZ1CK+jMn4=; b=JeigTEOr7SMqgnMuYOWPYNHpar2Z2tt34/E78fNC7zKGOjNu+TSuqcE9TS2prZHTx1 QhAQZoLbA7QVj+l65cOS3AuR39fgaAdVzqylYLjFRSVXZYPYGGpahF7Xnahs+jw7KGiO rPjAAc5s6TLCIXRusYY0ZYB6AUvMB8jn/JWwY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725353872; x=1725958672; h=list-unsubscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :message-id:date:subject:cc:to:from:x-beenthere:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=lzWjY0TH9hflGSMg7UGhKSz8LzpXyLI/jfZ1CK+jMn4=; b=bAxj+9V7xq1A3zq2OyoIBKZy8bqH7lWAPjfMrxQDOdZi+hks8RZgae3/Yr4pDOsWHr Dy2WQ6Tb3W//EXCXFKks9GDAjAMYwQyAt8nlTQ8t0U0uto8Ub1EM4JbeT7AblqCt0yBY fkFblvX9QpGIqWsvPfSRcZCE90C3hLLsIdjGTPxZYX23BJ/4jTeMUanm/doi+QozfkRk zOgnqyl/+lJp9B92jgYK4cLuoJKqmIUOMfSw9RauNto6XGTmOUtVYiPDDkxTpECWscln NWFa8YEjo11DvVFphgO5OnckND3lBaF9QHD7EtEN/pOnAfsA+WcdsRyayiVk7JLXTC9a a8xQ== X-Forwarded-Encrypted: i=2; AJvYcCVVlBIMrS/s2frtEOTj5ak68yi69gaDbu+3twJybkrU8aB5sY3ClF5hoOHMLmVY6L7hdPDfYqnYTCsdH2F8@patchwork.amarulasolutions.com X-Gm-Message-State: AOJu0Ywd2ACXqAcnw+A83InnEgd7Ge5KVoO6HzOXl7t35dhACC+cyTdb EXSCd7NUzv7gRrFCR4oaYRsz4pOfMssTv7aD+AiZsgxjq9LWhR7dmOlu2WcFAQitkg== X-Google-Smtp-Source: AGHT+IHpEl8i2YUbNb2SXyLprS8w6EmS4fWzmELhYaCFDmLUbDTL/KjeidQu/jTn5N7BrZryVGbdHQ== X-Received: by 2002:a05:6402:2353:b0:5c0:8f7c:ad9e with SMTP id 4fb4d7f45d1cf-5c21ed3e3e3mr16360603a12.15.1725353871740; Tue, 03 Sep 2024 01:57:51 -0700 (PDT) X-BeenThere: linux-amarula@amarulasolutions.com Received: by 2002:a05:6402:4301:b0:5be:ef89:e7ab with SMTP id 4fb4d7f45d1cf-5c21ec51831ls986923a12.2.-pod-prod-05-eu; Tue, 03 Sep 2024 01:57:50 -0700 (PDT) X-Received: by 2002:a05:6402:518b:b0:5c2:6d13:c583 with SMTP id 4fb4d7f45d1cf-5c26d13c80fmr1359225a12.28.1725353869753; Tue, 03 Sep 2024 01:57:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725353869; cv=none; d=google.com; s=arc-20160816; b=zt8t9viFkTLr/MNOT8Mu2ZX9FTGI67IcotIqkWAP5MvtOP2TQEw7lOdVsrdK7r0GcX A68fSyr6bTgWW2dr6slE6y4E5M34vCVNzzn1p7OaZYx70XGhQ16Kf9pINVcAmJYoCpqt WbhvcsgiX/CIGJO/8iQBMA4LR7yVsZHeyosw3IAT7dJyrT8YsQe/XYfaHXex/3dNA3I4 n/lfNEQH33Rgkl7pnTKncKD2CZNA8fGtUquWK+sSFB/vzahAkjVzTiibAM5FS3IkdkQv +mY+K7YOJAN/+Z7zTGd1O+1wY1puRbflAjoE0RyRGaqDJxNXq7mpJpDxL2AuUiftv7Ho TgAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dkim-signature; bh=teRRsZfFE9whINoPHQ2NNCQOMz3QDap3bmLVQ/9UIjU=; fh=tFw5bWiU1jBH/a1ncq8wme8lpRu4N+XMJNVYG1SrYIo=; b=qvpgKBIHsMcyrPQV7UOm6Hd2FrOz+rYvnF/Zseke4yGH0r9+tkDPOWdXD7Pz4knrpy xcMR01GLbGwtoz7hmsPTBdA31q/PVuCXxUkF0DmMD3um5rLrLXsTOj28YPqiWoa2xg0l QnLCvcyWhpfeo8SRFuAGMkItKTlY57gC0jkwDXqqX4LAzJTmhhjtutNcT1e3N0flrecJ IUSvN1E/Teauopn/kaz17vbQJr4CGj++CsHVFiqQ6/6m+ftQA3saFYlbgbhhhRkZb9fD qQvB2UkowkKhYmvXqzYFpsqmB8IqaKFtIRZnlJOrfiVAbPQzKDaByGLXmkBMfDcs6eHc hs8Q==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@amarulasolutions.com header.s=google header.b=YHvhtxTV; spf=pass (google.com: domain of michael@amarulasolutions.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=michael@amarulasolutions.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=amarulasolutions.com; dara=pass header.i=@amarulasolutions.com Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id 4fb4d7f45d1cf-5c26f1d9a8fsor83836a12.2.2024.09.03.01.57.49 for (Google Transport Security); Tue, 03 Sep 2024 01:57:49 -0700 (PDT) Received-SPF: pass (google.com: domain of michael@amarulasolutions.com designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; X-Received: by 2002:a05:6402:1d55:b0:5c2:6311:c9d1 with SMTP id 4fb4d7f45d1cf-5c26311cb25mr3683446a12.22.1725353868659; Tue, 03 Sep 2024 01:57:48 -0700 (PDT) Received: from panicking.amarulasolutions.com (93-35-133-65.ip55.fastwebnet.it. [93.35.133.65]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c226ccfeacsm6204487a12.78.2024.09.03.01.57.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Sep 2024 01:57:48 -0700 (PDT) From: Michael Trimarchi To: buildroot@buildroot.org Cc: linux-amarula@amarulasolutions.com, Michael Trimarchi Subject: [PATCH V2 1/2] scripts/cve: Avoid to do a complete clone of cve git repository Date: Tue, 3 Sep 2024 10:57:44 +0200 Message-ID: <20240903085745.2594893-1-michael@amarulasolutions.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-Original-Sender: michael@amarulasolutions.com X-Original-Authentication-Results: mx.google.com; dkim=pass header.i=@amarulasolutions.com header.s=google header.b=YHvhtxTV; spf=pass (google.com: domain of michael@amarulasolutions.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=michael@amarulasolutions.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=amarulasolutions.com; dara=pass header.i=@amarulasolutions.com Content-Type: text/plain; charset="UTF-8" Precedence: list Mailing-list: list linux-amarula@amarulasolutions.com; contact linux-amarula+owners@amarulasolutions.com List-ID: X-Spam-Checked-In-Group: linux-amarula@amarulasolutions.com X-Google-Group-Id: 476853432473 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , Just a simple clone and pull with --depth 1 should be enough to parse the cve and generate the pkg-stats report. From a full clone and a depth-1 clone, and the size delta is 2.9GiB vs. 2.2GiB. The download size does change: from 983.55MiB down to 270.78MiB. it's a net time win too: 2m17s vs 1min7s (on a 100Mbps link). Signed-off-by: Michael Trimarchi --- V1->V2: - Add statistics from Yann E. Morin - Use git pull --depth 1 for update the repo --- support/scripts/cve.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/support/scripts/cve.py b/support/scripts/cve.py index e25825581e..dcb3a63925 100755 --- a/support/scripts/cve.py +++ b/support/scripts/cve.py @@ -72,7 +72,7 @@ class CVE: print(f"Updating from {NVD_BASE_URL}") if os.path.exists(nvd_git_dir): subprocess.check_call( - ["git", "pull"], + ["git", "pull", "--depth", "1"], cwd=nvd_git_dir, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, @@ -82,7 +82,7 @@ class CVE: # happily clones into an empty directory. os.makedirs(nvd_git_dir) subprocess.check_call( - ["git", "clone", NVD_BASE_URL, nvd_git_dir], + ["git", "clone", "--depth", "1", NVD_BASE_URL, nvd_git_dir], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, ) From patchwork Tue Sep 3 08:57:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Trimarchi X-Patchwork-Id: 3361 Return-Path: X-Original-To: linux-amarula@patchwork.amarulasolutions.com Delivered-To: linux-amarula@patchwork.amarulasolutions.com Received: from mail-lj1-f197.google.com (mail-lj1-f197.google.com [209.85.208.197]) by ganimede.amarulasolutions.com (Postfix) with ESMTPS id 59B0D3F228 for ; Tue, 3 Sep 2024 10:57:53 +0200 (CEST) Received: by mail-lj1-f197.google.com with SMTP id 38308e7fff4ca-2f515891a64sf48572771fa.2 for ; Tue, 03 Sep 2024 01:57:53 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1725353873; cv=pass; d=google.com; s=arc-20240605; b=NPIQrkSOPRxaBnvyVivFOmxYvtY21TlFj3fk9WKrkifbOC5jUWLF0CHFR03A0dHCTq Lx2JKJ+V2+FcMFt4ZH27XVqm4XAuNkRhbkxCsSWidq29vx1jT735Zf8rEQ2T330VieLJ iBJiJ3kQ3fOPYMIUURfGe0ZWjeMO6f1C/xGBsgqI96cR7ItdDaGW2URiLNvh6MCM2BTr c9X+iaeBSvuvIEVVaB6tRAI2rc9pTEyG4HQeFEW8KWBBr5hm3KdO1oGG4iCsbKZj12YO fD9xRjjxuWup/8Uw4g0QCZXYrkK9Uh5bnq3Mp2DgbEq9zezODSarqDPlpK9toTONO6TH zPmg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-archive:list-help:list-post:list-id :mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=jQdc8/npVdo5yvyh6ZGZEkge/fW/SS8rNfI9hBU44eI=; fh=EbhNGqucF1ame+MhKUJ7jviRk4FQaXyqo+D3sc4fH3Y=; b=kumf+8Y9pkgqcPstY6ppfZE+G7FPdx2yBvxSJ2j0ZORvpbT7KBCLNE5F+IPErGPpNq JOZvoW/yWR71/Kx8TlY2xUYQGxtqFU8K/92OijRy1A2NfI2EKO317/gsdN2CX2YvdLIw hkX9AADBTnOoBdH2K7EwxanbxdcmRBqefKhRwbIlfkHPbNlfMJGb5HORNiZK3LQ7g50w bRk+yTvN1kbZ2flPHHcYk14Wj57eVE0M6OEtzm1yiyXdizdsMIUYz6/UymuAWXoQ/hvv gWGnr7ezTtdkg299ki75g4W+nEHuylPk3qchez6ikwaPQq00AVJXx+fAVNz+CzluFg9F MXbg==; darn=patchwork.amarulasolutions.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amarulasolutions.com header.s=google header.b=L9eawMVI; spf=pass (google.com: domain of michael@amarulasolutions.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=michael@amarulasolutions.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=amarulasolutions.com; dara=pass header.i=@amarulasolutions.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amarulasolutions.com; s=google; t=1725353872; x=1725958672; darn=patchwork.amarulasolutions.com; h=list-unsubscribe:list-archive:list-help:list-post:list-id :mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=jQdc8/npVdo5yvyh6ZGZEkge/fW/SS8rNfI9hBU44eI=; b=l/EEwLSafpr1c3OoP5LPsdGFSHa9dmz08bf1rO95dhdwN8N9LyMaFz+9J1F8nGie0p ThFWntYwE7MQ54xFqc32r0GqNa7R0M837oHrSW7e6N5P1CbIF1PvdGNBPO3PspafLiDv k2WbnlMWGFrDcls1gQJxA9uuO/I4X+R48Xmlk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725353872; x=1725958672; h=list-unsubscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=jQdc8/npVdo5yvyh6ZGZEkge/fW/SS8rNfI9hBU44eI=; b=F+REDWUIXgkLN9JZhyofWTASWMr+1QfbNrmhma0q26oOXbmfrkUfya8fiAdX9i9DUS GyrNdPE5mvFTX9nbMSM//P8bv6+gcSptdjADGQhWG1Y+LLRc1Xo+0tE/t/ATjJeVTLot U6tl2V7+12ZLoJ+aj3+VYtF7SWbfgUNHUFWD8xzuhAvRWsS4YNJlv0l9fiJQHkZGlipS lB3XkDRQBaTcV6866+62VPCu2UR0NCQ/uYHgHjRYbqXWFUbu66VN4JcT4YOzeQXseYYC TwpnqovFQ9YB8IHTrfKPvx+8KbLp6cQZ1/Png1B4oOG1Y184km8jQ01rcFpikQTJkZ3T LrFA== X-Forwarded-Encrypted: i=2; AJvYcCW3/3oSru+qNnNdVVW4/EyCqw1MeBzsOl1CHylhPaDr+9LVwb4R/IMrq/3FZpJw9uKlXhtOGPNRlZm7qEMl@patchwork.amarulasolutions.com X-Gm-Message-State: AOJu0Yxbu/x7T1LGiL0x6gWmKUdiNhE55iGgXTDbHQf1WUsiCczsrMDk fakPlZ1wOJbNNmqDIK7ukiN92KqzcgRcoVgWDp2e4HxPI7yUWHgeoSoUCsby7e6rbA== X-Google-Smtp-Source: AGHT+IGRaYzPkZ/l0k9K+EA1+V+bMn17Q8TXoRiA+tGGOJS88BazmBl5SghljZXli1AkuY+vtyVwAw== X-Received: by 2002:a2e:a781:0:b0:2f1:a19b:d5a4 with SMTP id 38308e7fff4ca-2f636a2b8f9mr33344161fa.22.1725353872499; Tue, 03 Sep 2024 01:57:52 -0700 (PDT) X-BeenThere: linux-amarula@amarulasolutions.com Received: by 2002:a05:6402:40c1:b0:5c2:61a7:62df with SMTP id 4fb4d7f45d1cf-5c261a76a38ls379450a12.0.-pod-prod-01-eu; Tue, 03 Sep 2024 01:57:51 -0700 (PDT) X-Received: by 2002:a05:6402:d06:b0:5c2:6bf7:8531 with SMTP id 4fb4d7f45d1cf-5c26bf78562mr1282281a12.33.1725353870590; Tue, 03 Sep 2024 01:57:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725353870; cv=none; d=google.com; s=arc-20160816; b=n7T2xzMsFQalNrl+T8X0ynJRBRnf0jbcov9Y2q+AwjFGw9AoxrQXWYjwVdk1nBvSwz SjahvQgzzblPW6/E9Now3bjs2QIeAyA5CrTtVBdkOMBMG/8ZDRSCsGyiNOfpq4KFyTTN T5n2lOnD64IEH3rWN1JBqP2gJABCO+U8ZurhbH8eadx3UXH4cMUK1F99WLvTHtlT1RI2 v9WK8K1JK/mAzSQ2OhfcpA4Q14NzyxRDDn45QjdB0kL8IXfle9lFkHda3px/922D9t6C AOPSb1cP7vzt8kdp2jICAXWFp0tgFhhdab6oNM+l/zJIUMaNHn2LkkjZC1ckx5aKBWUp bIPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=Z/tdhvFiWdgaMAbB6/frfY7NgEMt1w255SEMEObrNH8=; fh=tFw5bWiU1jBH/a1ncq8wme8lpRu4N+XMJNVYG1SrYIo=; b=WjjRwrzuTLjNhNUA3AaeHs2rwxcTVZPiOr8ePNLW5i4aTIpoC0YCYfzKZ7aibhiwKH nS8mWL4In98rJYL6mDyiqRvG+kdvSy6wTKJ/TF0SXumdFnxXoxwu4E1R9nEQD02RNI7c nvwmKyAMkuqSAIerrQkYfKVW8/c7jn3reNAHq80I7ubfbH0i7KwOZb2QGyQmibgmzDLK jrUNCGoNfuGLWv8I6qRCm74ju85fSGWvgrCaXuqmjkywkcenjV5JE4gqBCAdc7NzUNz6 8Rjy6E2SNmUb72nf0tiSN7SVv+zuC8PM9ji3ORa3J6Mv9ey/iU7bmbd4qtAs/z1IqHs+ Vbag==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@amarulasolutions.com header.s=google header.b=L9eawMVI; spf=pass (google.com: domain of michael@amarulasolutions.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=michael@amarulasolutions.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=amarulasolutions.com; dara=pass header.i=@amarulasolutions.com Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id 4fb4d7f45d1cf-5c25254cfd3sor820024a12.2.2024.09.03.01.57.50 for (Google Transport Security); Tue, 03 Sep 2024 01:57:50 -0700 (PDT) Received-SPF: pass (google.com: domain of michael@amarulasolutions.com designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; X-Received: by 2002:a05:6402:26c5:b0:5c2:4740:939d with SMTP id 4fb4d7f45d1cf-5c25c4034bamr4762882a12.26.1725353869700; Tue, 03 Sep 2024 01:57:49 -0700 (PDT) Received: from panicking.amarulasolutions.com (93-35-133-65.ip55.fastwebnet.it. [93.35.133.65]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c226ccfeacsm6204487a12.78.2024.09.03.01.57.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Sep 2024 01:57:49 -0700 (PDT) From: Michael Trimarchi To: buildroot@buildroot.org Cc: linux-amarula@amarulasolutions.com, Michael Trimarchi Subject: [PATCH V2 2/2] scripts/cve: Restart the clone if the pull generate an exception Date: Tue, 3 Sep 2024 10:57:45 +0200 Message-ID: <20240903085745.2594893-2-michael@amarulasolutions.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240903085745.2594893-1-michael@amarulasolutions.com> References: <20240903085745.2594893-1-michael@amarulasolutions.com> MIME-Version: 1.0 X-Original-Sender: michael@amarulasolutions.com X-Original-Authentication-Results: mx.google.com; dkim=pass header.i=@amarulasolutions.com header.s=google header.b=L9eawMVI; spf=pass (google.com: domain of michael@amarulasolutions.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=michael@amarulasolutions.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=amarulasolutions.com; dara=pass header.i=@amarulasolutions.com Content-Type: text/plain; charset="UTF-8" Precedence: list Mailing-list: list linux-amarula@amarulasolutions.com; contact linux-amarula+owners@amarulasolutions.com List-ID: X-Spam-Checked-In-Group: linux-amarula@amarulasolutions.com X-Google-Group-Id: 476853432473 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , If we are not able to pull from the directory, restart from a clean clone. This can happen for corrupt repository or unfinished download Signed-off-by: Michael Trimarchi --- V1->V2: Adjust the commit message --- support/scripts/cve.py | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/support/scripts/cve.py b/support/scripts/cve.py index dcb3a63925..6cd9aab963 100755 --- a/support/scripts/cve.py +++ b/support/scripts/cve.py @@ -21,6 +21,7 @@ import datetime import os import distutils.version import json +import shutil import subprocess import sys import operator @@ -69,15 +70,21 @@ class CVE: @staticmethod def download_nvd(nvd_git_dir): + done = False print(f"Updating from {NVD_BASE_URL}") if os.path.exists(nvd_git_dir): - subprocess.check_call( - ["git", "pull", "--depth", "1"], - cwd=nvd_git_dir, - stdout=subprocess.DEVNULL, - stderr=subprocess.DEVNULL, - ) - else: + try: + subprocess.check_call( + ["git", "pull", "--depth", "1"], + cwd=nvd_git_dir, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + ) + done = True + except: + shutil.rmtree(nvd_git_dir) + + if (not done): # Create the directory and its parents; git # happily clones into an empty directory. os.makedirs(nvd_git_dir)