Behind the public hardware database linux-hardware.org is basically one open source program: hw-probe. This little helper tool collects information from local hardware devices, tries to determine whether or not they are correctly working and uploads the "probe" into the database of linux-hardware.org.
This database is a great helper if you want to buy new hardware but Linux compatibility is not (well) documented of that hardware device. I've used this database to check Linux compatibility in Spring 2021 when my main workstation for daily work received a hardware upgrade (ASRock B550 and AMD Ryzen 5 3600 on Linux Mint 20.1 with Kernel 5.8).
To upload the probe, an Internet connection is obviously required. However there are many environments, where direct Internet access is not possible or blocked by a firewall. hw-probe then fails to upload the probe:
root@linux:~# sudo -E hw-probe -all -upload
Probe for hardware ... Ok
Reading logs ... WARNING: X11-related logs are not collected (try to run 'xhost +local:' to enable access or run as root by su)
Ok
read failed: Connection reset by peer at /usr/share/perl5/LWP/Protocol/http.pm line 382.
ERROR: failed to upload data
In such environments, an outgoing http/https proxy is often used. But even by defining the proxy using environment variables, the probe upload would still fail:
root@linux:~# export https_proxy=http://myproxy.example.com:3128
root@linux:~# export http_proxy=http://myproxy.example.com:3128
root@linux:~# sudo -E hw-probe -all -upload
Probe for hardware ... Ok
Reading logs ... WARNING: X11-related logs are not collected (try to run 'xhost +local:' to enable access or run as root by su)
Ok
read failed: Connection reset by peer at /usr/share/perl5/LWP/Protocol/http.pm line 382.
ERROR: failed to upload data
The errors reveal that in the background the Perl module LWP is used.
As /usr/bin/hw-probe is a Perl script, the source code can be read and modified (without having to run a ton of compilers). The relevant upload part indeed starts a new LWP::UserAgent connection in the script's postRequest function:
sub postRequest($$$)
{
my ($UploadURL, $Data, $SSL) = @_;
require LWP::UserAgent;
my $UAgent = LWP::UserAgent->new(parse_head => 0);
if($SSL eq "NoSSL" or not checkModule("Mozilla/CA.pm"))
{
$UploadURL=~s/\Ahttps:/http:/g;
$UAgent->ssl_opts(verify_hostname => 0, SSL_verify_mode => 0x00);
}
my $Res = $UAgent->post(
$UploadURL,
Content_Type => "form-data",
Content => $Data
);
my $Out = $Res->{"_content"};
if(not $Out) {
return $Res->{"_headers"}{"x-died"};
}
return $Out;
}
The LWP::UserAgent module also supports a proxy as option, but obviously this is not implemented here.
To add support for outgoing http/https proxies, I modified the postRequest function, checking whether or not the command line parameter -proxy (which is new) was used and then use the UserAgent option 'proxy':
sub postRequest($$$)
{
my ($UploadURL, $Data, $SSL) = @_;
require LWP::UserAgent;
my $UAgent = LWP::UserAgent->new(parse_head => 0);
if($Opt{"Proxy"}) {
my $proxy = $Opt{"Proxy"};
$UAgent->proxy([ [ 'http', 'https' ] => "http://$proxy" ]);
}
if($SSL eq "NoSSL" or not checkModule("Mozilla/CA.pm"))
{
$UploadURL=~s/\Ahttps:/http:/g;
$UAgent->ssl_opts(verify_hostname => 0, SSL_verify_mode => 0x00);
}
my $Res = $UAgent->post(
$UploadURL,
Content_Type => "form-data",
Content => $Data
);
my $Out = $Res->{"_content"};
if(not $Out) {
return $Res->{"_headers"}{"x-died"};
}
return $Out;
}
To see the full diff:
$ git diff
diff --git a/hw-probe.pl b/hw-probe.pl
index 3d1a084..8a2b0a2 100644
--- a/hw-probe.pl
+++ b/hw-probe.pl
@@ -167,6 +167,7 @@ GetOptions("h|help!" => \$Opt{"Help"},
"upload|confirm-upload-of-hashed-ids!" => \$Opt{"Upload"},
"hwinfo-path=s" => \$Opt{"HWInfoPath"},
"log!" => \$Opt{"ShowLog"},
+ "proxy=s" => \$Opt{"Proxy"},
# Inventory
"inventory|inventory-id|i|group|g=s" => \$Opt{"Group"},
"generate-inventory|generate-inventory-id|get-inventory-id|get-group!" => \$Opt{"GenerateGroup"},
@@ -355,6 +356,9 @@ GENERAL OPTIONS:
-hwinfo-path PATH
Path to a local hwinfo binary.
+ -proxy address:port
+ Set outgoing http/https proxy using syntax: proxy.domain.local:3128
+
INVENTORY OPTIONS:
-i|-inventory-id ID
Mark the probe by inventory ID.
@@ -2475,6 +2479,11 @@ sub postRequest($$$)
require LWP::UserAgent;
my $UAgent = LWP::UserAgent->new(parse_head => 0);
+
+ if($Opt{"Proxy"}) {
+ my $proxy = $Opt{"Proxy"};
+ $UAgent->proxy([ [ 'http', 'https' ] => "http://$proxy" ]);
+ }
if($SSL eq "NoSSL" or not checkModule("Mozilla/CA.pm"))
{
The changes were pushed to the upstream project as Pull Request #102.
Running hw-probe now with the new -proxy parameter and the proxy defined in the command line:
root@linux:~# sudo -E hw-probe -all -upload -proxy myproxy.example.com:3128
Probe for hardware ... Ok
Reading logs ... WARNING: X11-related logs are not collected (try to run 'xhost +local:' to enable access or run as root by su)
Ok
Uploaded to DB, Thank you!
Probe URL: https://linux-hardware.org/?probe=a35e7d3eb0
And the probe was sucessfully uploaded!
Open Source Rulez!
AWS Android Ansible Apache Apple Atlassian BSD Backup Bash Bluecoat CMS Chef Cloud Coding Consul Container Containers CouchDB DB DNS Database Databases Docker ELK Elasticsearch Filebeat FreeBSD Galera GlusterFS Grafana Graphics HAProxy HTML Hacks Hardware Icinga Icingaweb2 InfluxDB Internet Java KVM Kibana Kodi Kubernetes LTS LXC Linux Logstash Mac Macintosh Mail MariaDB Minio MongoDB Monitoring Multimedia MySQL NFS Nagios Network Nginx OSSEC OTRS Office PGSQL PHP Perl Personal PostgreSQL Postgres PowerDNS Proxmox Proxy Python Rancher Redis SSL Samba Seafile Security Shell SmartOS Solaris Surveillance SystemD Systemd TLS Tomcat Ubuntu Unix VMWare VMware Varnish Virtualization Windows Wireless Wordpress Wyse ZFS Zoneminder