project/uclient.git
3 months agotests: fix help usage test for uclient built with sanitizer
Petr Štetiar [Sun, 4 Apr 2021 07:07:41 +0000 (09:07 +0200)]
tests: fix help usage test for uclient built with sanitizer

Commit 64e00d6db4c3 ("uclient-fetch: document missing options") has
changed output of usage help so adjust test accordingly.

Fixes: 64e00d6db4c3 ("uclient-fetch: document missing options")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
3 months agotests: fix help usage test
Petr Štetiar [Sun, 4 Apr 2021 07:07:41 +0000 (09:07 +0200)]
tests: fix help usage test

Commit 64e00d6db4c3 ("uclient-fetch: document missing options") has
changed output of usage help so adjust test accordingly.

Fixes: 64e00d6db4c3 ("uclient-fetch: document missing options")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
3 months agouclient-fetch: document missing options
Baptiste Jonglez [Wed, 30 Dec 2020 22:02:41 +0000 (23:02 +0100)]
uclient-fetch: document missing options

Missing ones were: --no-proxy, --quiet, --continue / -c

Also improved readability of several options.

Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
4 months agotests: fix possibly longer start of HTTP server
Petr Štetiar [Mon, 22 Mar 2021 20:16:51 +0000 (21:16 +0100)]
tests: fix possibly longer start of HTTP server

Seems like sometimes it can take some time to start HTTP server so let's
wait for the server becoming operating before executing test cases.

References: https://gitlab.com/openwrt/project/uclient/-/jobs/1118330555#L984
Signed-off-by: Petr Štetiar <ynezz@true.cz>
7 months agoAdd initial GitLab CI support
Petr Štetiar [Tue, 8 Dec 2020 11:36:18 +0000 (12:36 +0100)]
Add initial GitLab CI support

Uses currently proof-of-concept openwrt-ci[1] in order to:

 * improve the quality of the codebase in various areas
 * decrease code review time and help merging contributions faster
 * get automagic feedback loop on various platforms and tools
   - out of tree build with OpenWrt SDK on following targets:
     * ath79-generic
     * imx6-generic
     * malta-be
     * mvebu-cortexa53
   - out of tree native build on x86/64 with GCC (versions 8, 9, 10) and Clang 10
   - out of tree native x86/64 static code analysis with cppcheck and
     scan-build from Clang 10

1. https://gitlab.com/ynezz/openwrt-ci/

Signed-off-by: Petr Štetiar <ynezz@true.cz>
7 months agouclient-fetch: wolfSSL: fix certificate validation
Petr Štetiar [Thu, 10 Dec 2020 09:58:55 +0000 (10:58 +0100)]
uclient-fetch: wolfSSL: fix certificate validation

Currently wolfSSL doesn't validate any certificates, quoting from
README:

 wolfSSL takes a different approach to certificate verification than
 OpenSSL does. The default policy for the client is to verify the server,
 this means that if you don't load CAs to verify the server you'll get a
 connect error, no signer error to confirm failure (-188).

 If you want to mimic OpenSSL behavior of having SSL_connect succeed even if
 verifying the server fails and reducing security you can do this by calling:

 wolfSSL_CTX_set_verify(ctx, SSL_VERIFY_NONE, 0);

 before calling wolfSSL_new();. Though it's not recommended.

wolfSSL simply behaves differently then OpenSSL so once you set
SSL_VERIFY_NONE wolfSSL doesn't care about the certificates anymore so
every call to SSL_get_verify_result() is going to succeed (returns
X509_V_OK) even for invalid certificates and current OpenSSL based post
connection verification logic thus doesn't work.

So in order to get the validation working we need to use SSL_VERIFY_PEER
for wolfSSL by default and allow disabling it explicitly by new
`context_set_require_validation()` call.

Fixes: FS#3465
Signed-off-by: Petr Štetiar <ynezz@true.cz>
7 months agouclient-fetch: init_ca_cert: fix memory leak
Petr Štetiar [Tue, 8 Dec 2020 17:54:57 +0000 (18:54 +0100)]
uclient-fetch: init_ca_cert: fix memory leak

Fixes following memory leak:

 $ valgrind --quiet --leak-check=full uclient-fetch -q http://127.0.0.1:1922/does-not-exist

  51 (16 direct, 35 indirect) bytes in 1 blocks are definitely lost in loss record 4 of 9
   at 0x4C31A3F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
   by 0x4C33D84: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
   by 0x553C7DF: glob_in_dir (glob.c:1410)
   by 0x553D5E8: glob@@GLIBC_2.27 (glob.c:1097)
   by 0x401D62: init_ca_cert (uclient-fetch.c:503)
   by 0x401D62: main (uclient-fetch.c:741)

Signed-off-by: Petr Štetiar <ynezz@true.cz>
7 months agocmake: enable extra compiler checks
Petr Štetiar [Tue, 8 Dec 2020 16:23:36 +0000 (17:23 +0100)]
cmake: enable extra compiler checks

Let's enforce additional automatic checks enforced by the compiler in
order to catch possible errors during compilation.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
7 months agouclient-http: fix extra compiler warnings on mips_24kc and cortex-a9+neon
Petr Štetiar [Tue, 8 Dec 2020 17:13:03 +0000 (18:13 +0100)]
uclient-http: fix extra compiler warnings on mips_24kc and cortex-a9+neon

Fixes following warnings as reported on 32-bit platforms
toolchain-mips_24kc_gcc-8.4.0_musl and toolchain-arm_cortex-a9+neon_gcc-8.4.0_musl_eabi:

 uclient-http.c:1111:10: error: comparison of integer expressions of different signedness: 'unsigned int' and 'int' [-Werror=sign-compare]
   if (len > data_end - data)
           ^
 uclient-http.c:1115:11: error: comparison of integer expressions of different signedness: 'unsigned int' and 'long int' [-Werror=sign-compare]
    if (len > uh->read_chunked)
            ^
 uclient-http.c:1120:11: error: comparison of integer expressions of different signedness: 'unsigned int' and 'long int' [-Werror=sign-compare]
    if (len > uh->content_length)
            ^

References: https://gitlab.com/ynezz/openwrt-uclient/-/pipelines/226912126
Signed-off-by: Petr Štetiar <ynezz@true.cz>
7 months agouclient-fetch: fix potential memory leaks
Petr Štetiar [Tue, 8 Dec 2020 16:45:19 +0000 (17:45 +0100)]
uclient-fetch: fix potential memory leaks

Fixes following issue reported by clang-12 static analyzer:

 uclient-fetch.c:612:25: warning: Potential leak of memory pointed to by 'username' [unix.Malloc]
                                 memset(optarg, '*', strlen(optarg));
                                                     ^~~~~~~~~~~~~~
 uclient-fetch.c:618:25: warning: Potential leak of memory pointed to by 'password' [unix.Malloc]
                                 memset(optarg, '*', strlen(optarg));
                                                     ^~~~~~~~~~~~~~

Signed-off-by: Petr Štetiar <ynezz@true.cz>
7 months agouclient: fix initialized but never read variable
Petr Štetiar [Tue, 8 Dec 2020 16:41:44 +0000 (17:41 +0100)]
uclient: fix initialized but never read variable

Fixes following issue reported by clang-12 static analyzer:

 uclient.c:290:22: warning: Value stored to 'url' during its initialization is never read [deadcode.DeadStores]
         struct uclient_url *url = cl->url;
                             ^~~   ~~~~~~~

Signed-off-by: Petr Štetiar <ynezz@true.cz>
7 months agouclient-fetch: fix statement may fallt hrough
Petr Štetiar [Tue, 8 Dec 2020 16:37:52 +0000 (17:37 +0100)]
uclient-fetch: fix statement may fallt hrough

Fixes following issue reported by clang-12 static analyzer:

 uclient-fetch.c:228:6: error: this statement may fall through [-Werror=implicit-fallthrough=]
    if (sscanf(blobmsg_get_string(tb[H_RANGE]),
       ^
 uclient-fetch.c:236:2: note: here
   case 204:
   ^~~~

Signed-off-by: Petr Štetiar <ynezz@true.cz>
7 months agouclient-http: fix freeing of stack allocated memory
Petr Štetiar [Tue, 8 Dec 2020 16:34:50 +0000 (17:34 +0100)]
uclient-http: fix freeing of stack allocated memory

Fixes following issue reported by clang-12 static analyzer:

 uclient-http.c:568:2: warning: Memory allocated by alloca() should not be deallocated [unix.Malloc]
         free(buf_orig);
         ^~~~~~~~~~~~~~

Signed-off-by: Petr Štetiar <ynezz@true.cz>
7 months agoFix extra compiler warnings
Petr Štetiar [Tue, 8 Dec 2020 16:29:57 +0000 (17:29 +0100)]
Fix extra compiler warnings

Fixes following -Wextra compiler warnings:

 uclient.c:195:16: error: comparison of integers of different signs: 'int' and 'unsigned long' [-Werror,-Wsign-compare]
         for (i = 0; i < ARRAY_SIZE(backends); i++) {
                     ~ ^ ~~~~~~~~~~~~~~~~~~~~

 uclient-http.c:619:2: error: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'int' [-Werror,-Wsign-compare]
         blobmsg_for_each_attr(cur, uh->headers.head, rem)
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 uclient-http.c:619:2: error: comparison of integers of different signs: 'int' and 'unsigned long' [-Werror,-Wsign-compare]
         blobmsg_for_each_attr(cur, uh->headers.head, rem)
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 uclient-http.c:993:16: error: comparison of integers of different signs: 'int' and 'unsigned long' [-Werror,-Wsign-compare]
         for (i = 0; i < ARRAY_SIZE(request_types); i++) {
                     ~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~

 uclient.c:195:16: error: comparison of integers of different signs: 'int' and 'unsigned long' [-Werror,-Wsign-compare]
         for (i = 0; i < ARRAY_SIZE(backends); i++) {
                     ~ ^ ~~~~~~~~~~~~~~~~~~~~

 uclient-http.c:619:2: error: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'int' [-Werror,-Wsign-compare]
         blobmsg_for_each_attr(cur, uh->headers.head, rem)
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 uclient-http.c:619:2: error: comparison of integers of different signs: 'int' and 'unsigned long' [-Werror,-Wsign-compare]
         blobmsg_for_each_attr(cur, uh->headers.head, rem)
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 uclient-http.c:993:16: error: comparison of integers of different signs: 'int' and 'unsigned long' [-Werror,-Wsign-compare]
         for (i = 0; i < ARRAY_SIZE(request_types); i++) {
                     ~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~

 uclient-fetch.c:551:67: error: missing field 'flag' initializer [-Werror,-Wmissing-field-initializers]
         [L_NO_CHECK_CERTIFICATE] = { "no-check-certificate", no_argument },

Signed-off-by: Petr Štetiar <ynezz@true.cz>
7 months agoAdd basic cram based unit tests
Petr Štetiar [Tue, 8 Dec 2020 11:29:59 +0000 (12:29 +0100)]
Add basic cram based unit tests

For the start just basic uclient-fetch functionality coverage.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
7 months agocmake: fix building out of the tree
Petr Štetiar [Tue, 8 Dec 2020 09:06:29 +0000 (10:06 +0100)]
cmake: fix building out of the tree

When building out of the tree, linker is unable to find the ubox library
so fix it by using find_library CMake command.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
13 months agouclient-fetch: add option to read POST data from file
Gio [Fri, 12 Jun 2020 10:59:52 +0000 (12:59 +0200)]
uclient-fetch: add option to read POST data from file

Passing post data in command line is convenient but has limited size,
and may become tricky to correctly escape passed data especially in
scripts.
This patch add the option --post-file so the data to post can be read
from a file.
Tested on x86/64.

Acked-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Gioacchino Mazzurco <gio@eigenlab.org>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
17 months agouclient-fetch: support specifying advertised TLS ciphers
Jo-Philipp Wich [Sat, 15 Feb 2020 21:39:30 +0000 (22:39 +0100)]
uclient-fetch: support specifying advertised TLS ciphers

Introduce a new `--ciphers` option which allows specifying a colon separated
list of usable TLS ciphers.

Depending on the underlying ustream-ssl provider, the list either follows
OpenSSL's cipher string format or, in case of mbedTLS, is a simple colon
separated cipher whitelist.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
19 months agouclient: Add string error function
Daniel Danzberger [Thu, 5 Dec 2019 16:51:58 +0000 (17:51 +0100)]
uclient: Add string error function

This add's the uclient_strerror function, which resolves and error code
to a string message.

Signed-off-by: Daniel Danzberger <daniel@dd-wrt.com>
2 years agouclient-http: set data_eof when content-length is 0
Yousong Zhou [Mon, 27 May 2019 02:31:57 +0000 (02:31 +0000)]
uclient-http: set data_eof when content-length is 0

Otherwise uclient-fetch can report "Connection reset prematurely"

Fixes FS#2222

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
Acked-by: Petr Štetiar <ynezz@true.cz>
2 years agouclient-http: properly handle HTTP redirects via proxy connections
Jo-Philipp Wich [Sat, 24 Nov 2018 18:54:07 +0000 (19:54 +0100)]
uclient-http: properly handle HTTP redirects via proxy connections

When uclient connects via an HTTP proxy connection we need to swap the
contexts .proxy_url and not the .url property (which is the connection
URL to the proxy server itself).

This fixes uclient erroring out in a redirect loop when connecting via
a proxy connection.

Ref: https://forum.openwrt.org/t/opkg-update-fails/25813
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2 years agouclient-http: Close ustream file handle only if allocated
Tobias Schramm [Fri, 3 Aug 2018 21:39:21 +0000 (23:39 +0200)]
uclient-http: Close ustream file handle only if allocated

Since the connection setup in uclient_do_connect can fail before
ustream_init_fd is called we must check the fd was actually allocated
before closing it, else we would close STDIN.

Signed-off-by: Tobias Schramm <tobleminer@gmail.com>
3 years agouclient-utils: Handle memory allocation failure for url file name
Tobias Schramm [Tue, 20 Feb 2018 10:26:31 +0000 (11:26 +0100)]
uclient-utils: Handle memory allocation failure for url file name

Add null pointer check to allocation of url filename

Signed-off-by: Tobias Schramm <tobleminer@gmail.com>
3 years agouclient-http: Implement error handling for header-sending
Tobias Schramm [Tue, 20 Feb 2018 10:26:29 +0000 (11:26 +0100)]
uclient-http: Implement error handling for header-sending

Add some error handling for errors during assembly of headers

Signed-off-by: Tobias Schramm <tobleminer@gmail.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>
3 years agouclient: Handle memory allocation failure for url
Tobias Schramm [Sun, 18 Feb 2018 12:46:10 +0000 (13:46 +0100)]
uclient: Handle memory allocation failure for url

Check ptr to url after allocation to prevent null pointer
dereference

Signed-off-by: Tobias Schramm <tobleminer@gmail.com>
3 years agouclient-http: Handle memory allocation failure
Tobias Schramm [Sun, 18 Feb 2018 12:46:08 +0000 (13:46 +0100)]
uclient-http: Handle memory allocation failure

Add null pointer check to allocation of uclient_http struct

Signed-off-by: Tobias Schramm <tobleminer@gmail.com>
3 years agouclient-http: auth digest: Handle multiple possible memory allocation failures
Tobias Schramm [Sun, 18 Feb 2018 12:46:06 +0000 (13:46 +0100)]
uclient-http: auth digest: Handle multiple possible memory allocation failures

Add null pointer checks to allocation of buffers for authentication parameters

Signed-off-by: Tobias Schramm <tobleminer@gmail.com>
3 years agouclient-http: basic auth: Handle memory allocation failure
Tobias Schramm [Sun, 18 Feb 2018 12:46:05 +0000 (13:46 +0100)]
uclient-http: basic auth: Handle memory allocation failure

Allocation of the base64 buffer might fail, resulting in a null
ptr being passed to base64_encode as a target buffer

Signed-off-by: Tobias Schramm <tobleminer@gmail.com>
3 years agouclient-fetch: Check for nullpointer returned by uclient_get_url_filename
Tobias Schramm [Sun, 18 Feb 2018 12:46:04 +0000 (13:46 +0100)]
uclient-fetch: Check for nullpointer returned by uclient_get_url_filename

uclient_get_url_filename may return a null ptr resulting in a null
ptr dereference by fprintf and open

Signed-off-by: Tobias Schramm <tobleminer@gmail.com>
3 years agouclient-fetch: use package name pattern in message for missing SSL library
Michael Heimpold [Thu, 25 Jan 2018 20:53:45 +0000 (21:53 +0100)]
uclient-fetch: use package name pattern in message for missing SSL library

When faced with this error message, I did not directly know what
package(s) to install. I first grepped the package list for packages
named libustream-ssl... found nothing, and finally realized, that here
only the library name was mentioned.

But when we mention the certificate packages, why not also directly
mention the SSL/TLS packages with a pattern. That will save one step
next time when stumbling here - at least for me ;-)

Signed-off-by: Michael Heimpold <mhei@heimpold.de>
3 years agouclient-fetch: fix overloading of output_file variable
Felix Fietkau [Thu, 2 Nov 2017 20:58:42 +0000 (21:58 +0100)]
uclient-fetch: fix overloading of output_file variable

When uclient-fetch is called with multiple URL's, it derives the
first filename from the URL. It then sets the global output_file
variable, causing a use-after-free bug on an attempt to use it as output
file for the next file.

Fix this by avoiding the overwrite entirely by only setting a local
variable

Reported-by: Philip Prindeville <philipp@redfish-solutions.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>
3 years agouclient-http: fix Host: header for literal IPv6 addresses
Matthias Schiffer [Tue, 5 Sep 2017 18:02:37 +0000 (20:02 +0200)]
uclient-http: fix Host: header for literal IPv6 addresses

For literal IPv6 addresses, the host header must have the form

  Host: [...]

including the square brackets, as it may also contain a port. Some
webservers ignore the missing brackets, while others will return error 400.

IPv6 addresses are determined by searching for ':' characters in the host,
as neither IPv4 addresses nor DNS names can contain colons. An alternative
would be to add a flag to the uclient_url struct; but as this struct is
exposed in public headers, such a change could be considered an ABI change,
making a backport difficult.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Acked-by: Jo-Philipp Wich <jo@mein.io>
4 years agouclient-fetch: read_data_cb: fix a potential buffer overflow
Alexander Couzens [Sun, 11 Jun 2017 11:01:06 +0000 (13:01 +0200)]
uclient-fetch: read_data_cb: fix a potential buffer overflow

When a backend->read() is not implement or returns -1,
This -1 is interpreted as unsigned int resulting
in an access far behind the buffer.

Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
4 years agoremove obsolete mac os x /opt/local include/library search path
Felix Fietkau [Fri, 9 Dec 2016 12:06:59 +0000 (13:06 +0100)]
remove obsolete mac os x /opt/local include/library search path

Where necessary, this can easily be provided through CFLAGS/LDFLAGS

Signed-off-by: Felix Fietkau <nbd@nbd.name>
4 years agoFix unused results warnings
Florian Fainelli [Thu, 8 Dec 2016 03:04:45 +0000 (19:04 -0800)]
Fix unused results warnings

Fixes:

uclient-http.c:385:8: error: ignoring return value of 'fread', declared with attribute warn_unused_result [-Werror=unused-result]
   fread(&val, sizeof(val), 1, f);
        ^

uclient-fetch.c: In function 'main':
uclient-fetch.c:664:12: error: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Werror=unused-result]
    asprintf(&auth_str, "%s:%s", username, password);
            ^
uclient-fetch.c: In function 'read_data_cb':
uclient-fetch.c:269:9: error: ignoring return value of 'write', declared with attribute warn_unused_result [-Werror=unused-result]
    write(output_fd, buf, len);

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
4 years agouclient-http: send correct "Host:" header if port is set
Alexander Couzens [Wed, 30 Nov 2016 17:49:10 +0000 (18:49 +0100)]
uclient-http: send correct "Host:" header if port is set

When connecting to a website with a special port, uclient-fetch connects
to the correct port, but is sending an incorrect Host: header without
the corresponding port.

Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
4 years agoFix printf format in previous commit.
Jo-Philipp Wich [Sat, 30 Jul 2016 13:35:10 +0000 (15:35 +0200)]
Fix printf format in previous commit.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
4 years agoMake missing SSL support message more explicit.
Jo-Philipp Wich [Sat, 30 Jul 2016 13:33:52 +0000 (15:33 +0200)]
Make missing SSL support message more explicit.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
5 years agohttp: add support for relative location on redirect
Felix Fietkau [Tue, 5 Jul 2016 10:39:01 +0000 (12:39 +0200)]
http: add support for relative location on redirect

Signed-off-by: Felix Fietkau <nbd@nbd.name>
5 years agouclient-fetch: add support for --quiet
Felix Fietkau [Tue, 5 Jul 2016 09:19:38 +0000 (11:19 +0200)]
uclient-fetch: add support for --quiet

Signed-off-by: Felix Fietkau <nbd@nbd.name>
5 years agocmake: Find libubox/ustream-ssl.h header file
Florian Fainelli [Fri, 1 Jul 2016 23:03:47 +0000 (16:03 -0700)]
cmake: Find libubox/ustream-ssl.h header file

Add a CMake FIND_PATH and INCLUDE_DIRECTORIES searching for
libubox/ustream-ssl.h.  Some external toolchains which do not include
standard locations would fail to find the header otherwise.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
5 years agohttp: allow sending message body for DELETE request
Rafał Miłecki [Thu, 16 Jun 2016 12:10:05 +0000 (14:10 +0200)]
http: allow sending message body for DELETE request

Sending entity within DELETE is not forbidden by RFC 7231, see section
4.3.5. DELETE:
> A payload within a DELETE request message has no defined semantics;
> sending a payload body on a DELETE request might cause some existing
> implementations to reject the request.

Signed-off-by: Rafał Miłecki <zajec5@gmail.com>
5 years agouclient-fetch: fix inconsistencies in help text (thx, Hannu Nyman)
Felix Fietkau [Mon, 8 Feb 2016 18:03:57 +0000 (19:03 +0100)]
uclient-fetch: fix inconsistencies in help text (thx, Hannu Nyman)

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agohttp: use usock_inet_timeout()
Felix Fietkau [Thu, 28 Jan 2016 11:19:54 +0000 (12:19 +0100)]
http: use usock_inet_timeout()

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agohttp: add support for specifying ipv4/ipv6 preference
Felix Fietkau [Sun, 24 Jan 2016 23:18:33 +0000 (00:18 +0100)]
http: add support for specifying ipv4/ipv6 preference

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: truncate output files (unless resuming)
Felix Fietkau [Sat, 23 Jan 2016 19:59:45 +0000 (20:59 +0100)]
uclient-fetch: truncate output files (unless resuming)

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agohttp: assume data EOF if the connection terminates
Felix Fietkau [Sat, 23 Jan 2016 18:59:25 +0000 (19:59 +0100)]
http: assume data EOF if the connection terminates

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: fix progress bar initialization with -O-
Felix Fietkau [Sat, 23 Jan 2016 18:52:58 +0000 (19:52 +0100)]
uclient-fetch: fix progress bar initialization with -O-

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agohttp: fix processing server http data separated by \n instead of \r\n
Felix Fietkau [Sat, 23 Jan 2016 18:49:55 +0000 (19:49 +0100)]
http: fix processing server http data separated by \n instead of \r\n

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agohttp: add proper error handling to uclient_http_redirect()
Felix Fietkau [Sat, 23 Jan 2016 00:23:00 +0000 (01:23 +0100)]
http: add proper error handling to uclient_http_redirect()

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: allow overwriting files if -O was used
Felix Fietkau [Thu, 21 Jan 2016 15:58:26 +0000 (16:58 +0100)]
uclient-fetch: allow overwriting files if -O was used

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: fix short command line option for timeout
Felix Fietkau [Wed, 20 Jan 2016 09:52:40 +0000 (10:52 +0100)]
uclient-fetch: fix short command line option for timeout

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agohttp: fix connection close handling on auth
Felix Fietkau [Wed, 20 Jan 2016 09:51:58 +0000 (10:51 +0100)]
http: fix connection close handling on auth

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: add missing newline after final progress bar update
Felix Fietkau [Tue, 19 Jan 2016 21:59:02 +0000 (22:59 +0100)]
uclient-fetch: add missing newline after final progress bar update

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agocore: fix unused but set variable
Felix Fietkau [Sun, 17 Jan 2016 12:39:51 +0000 (13:39 +0100)]
core: fix unused but set variable

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agocore: make __uclient_get_url static
Felix Fietkau [Sun, 17 Jan 2016 12:39:23 +0000 (13:39 +0100)]
core: make __uclient_get_url static

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: add progress bar support
Felix Fietkau [Sun, 17 Jan 2016 12:37:59 +0000 (13:37 +0100)]
uclient-fetch: add progress bar support

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: add support for setting output directory
Felix Fietkau [Sat, 16 Jan 2016 22:39:10 +0000 (23:39 +0100)]
uclient-fetch: add support for setting output directory

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-http: use blocking connect, switch to non-blocking later. fixes ipv6 -> ipv4...
Felix Fietkau [Sat, 16 Jan 2016 22:29:51 +0000 (23:29 +0100)]
uclient-http: use blocking connect, switch to non-blocking later. fixes ipv6 -> ipv4 fallback

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agohttp: detect connect failure
Felix Fietkau [Sat, 16 Jan 2016 22:26:32 +0000 (23:26 +0100)]
http: detect connect failure

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: ignore SIGPIPE
Felix Fietkau [Sat, 16 Jan 2016 22:25:38 +0000 (23:25 +0100)]
uclient-fetch: ignore SIGPIPE

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: add proxy support
Felix Fietkau [Sat, 16 Jan 2016 22:08:55 +0000 (23:08 +0100)]
uclient-fetch: add proxy support

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agohttp: add proxy support
Felix Fietkau [Sat, 16 Jan 2016 22:08:46 +0000 (23:08 +0100)]
http: add proxy support

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agocore: add proxy support
Felix Fietkau [Sat, 16 Jan 2016 22:08:31 +0000 (23:08 +0100)]
core: add proxy support

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agocore: fix memory leak if url change fails
Felix Fietkau [Sat, 16 Jan 2016 19:13:36 +0000 (20:13 +0100)]
core: fix memory leak if url change fails

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-http: make username/password variables local
Felix Fietkau [Sat, 16 Jan 2016 18:46:33 +0000 (19:46 +0100)]
uclient-http: make username/password variables local

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: add support for resuming downloads
Felix Fietkau [Sat, 16 Jan 2016 18:34:22 +0000 (19:34 +0100)]
uclient-fetch: add support for resuming downloads

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: clean up output file selection
Felix Fietkau [Sat, 16 Jan 2016 18:18:38 +0000 (19:18 +0100)]
uclient-fetch: clean up output file selection

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agohttp: allow the header_done callback to reset the client state
Felix Fietkau [Sat, 16 Jan 2016 17:50:22 +0000 (18:50 +0100)]
http: allow the header_done callback to reset the client state

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: add support for --timeout
Felix Fietkau [Sat, 16 Jan 2016 12:08:45 +0000 (13:08 +0100)]
uclient-fetch: add support for --timeout

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: add support for --spider
Felix Fietkau [Sat, 16 Jan 2016 11:55:36 +0000 (12:55 +0100)]
uclient-fetch: add support for --spider

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: add support for --post-data
Felix Fietkau [Sat, 16 Jan 2016 11:50:11 +0000 (12:50 +0100)]
uclient-fetch: add support for --post-data

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: only add default CA entries if no certificate was supplied and verify...
Felix Fietkau [Sat, 16 Jan 2016 11:22:00 +0000 (12:22 +0100)]
uclient-fetch: only add default CA entries if no certificate was supplied and verify was requested

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: add support for setting the user agent
Felix Fietkau [Sat, 16 Jan 2016 11:20:09 +0000 (12:20 +0100)]
uclient-fetch: add support for setting the user agent

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: add support for fetching multiple urls
Felix Fietkau [Sat, 16 Jan 2016 10:43:40 +0000 (11:43 +0100)]
uclient-fetch: add support for fetching multiple urls

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: add user/password to command line help
Felix Fietkau [Sat, 16 Jan 2016 10:01:58 +0000 (11:01 +0100)]
uclient-fetch: add user/password to command line help

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: load CA certificates
Felix Fietkau [Sat, 16 Jan 2016 00:17:33 +0000 (01:17 +0100)]
uclient-fetch: load CA certificates

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agohttp: get remote address from usock instead of using getpeername
Felix Fietkau [Fri, 15 Jan 2016 20:30:09 +0000 (21:30 +0100)]
http: get remote address from usock instead of using getpeername

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
5 years agouclient-fetch: set server_name of the ssl context to support SNI
Alexander Couzens [Thu, 6 Aug 2015 11:02:47 +0000 (13:02 +0200)]
uclient-fetch: set server_name of the ssl context to support SNI

Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
6 years agouclient-http: end custom header lines with CRLF
Felix Fietkau [Tue, 14 Apr 2015 21:04:32 +0000 (23:04 +0200)]
uclient-http: end custom header lines with CRLF

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
6 years agouclient-http: allow .header_done() callback to reset the connection
Felix Fietkau [Tue, 14 Apr 2015 21:01:53 +0000 (23:01 +0200)]
uclient-http: allow .header_done() callback to reset the connection

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
6 years agoallow sending requests with DELETE method
Rafał Miłecki [Thu, 22 Jan 2015 10:45:13 +0000 (11:45 +0100)]
allow sending requests with DELETE method

Signed-off-by: Rafał Miłecki <zajec5@gmail.com>
6 years agouse const for char buffer in uclient_write
Rafał Miłecki [Wed, 21 Jan 2015 21:48:35 +0000 (22:48 +0100)]
use const for char buffer in uclient_write

We are not supposed to modify it and ustream accepts const already.

Signed-off-by: Rafał Miłecki <zajec5@gmail.com>
6 years agosupport for connection timeout
Rafał Miłecki [Fri, 16 Jan 2015 10:21:02 +0000 (11:21 +0100)]
support for connection timeout

Signed-off-by: Rafał Miłecki <zajec5@gmail.com>
6 years agohttp: fix processing of digest auth parameters
Felix Fietkau [Fri, 5 Dec 2014 10:56:18 +0000 (11:56 +0100)]
http: fix processing of digest auth parameters

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
6 years agohttp: terminate old connection in uclient_http_connect after incomplete request proce...
Felix Fietkau [Thu, 4 Sep 2014 11:19:59 +0000 (13:19 +0200)]
http: terminate old connection in uclient_http_connect after incomplete request processing

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
6 years agohttp: do not call uclient_http_disconnect from uclient_notify_eof directly
Felix Fietkau [Thu, 4 Sep 2014 11:10:59 +0000 (13:10 +0200)]
http: do not call uclient_http_disconnect from uclient_notify_eof directly

Depending on the context, it can lead to crashes

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
6 years agouclient-http: only invoke data_sent callback if it is provided
Jo-Philipp Wich [Mon, 28 Jul 2014 11:58:02 +0000 (13:58 +0200)]
uclient-http: only invoke data_sent callback if it is provided

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>
6 years agohttp: ignore empty lines before HTTP response
Felix Fietkau [Thu, 21 Aug 2014 19:10:30 +0000 (21:10 +0200)]
http: ignore empty lines before HTTP response

Fixes HTTP keepalive issues with some servers

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
7 years agofetch: fix segfault after destination was not reached
Luka Perkov [Mon, 7 Jul 2014 08:06:24 +0000 (10:06 +0200)]
fetch: fix segfault after destination was not reached

Signed-off-by: Luka Perkov <luka@openwrt.org>
7 years agohttp: implement data_sent write callback
Luka Perkov [Mon, 7 Jul 2014 08:11:41 +0000 (10:11 +0200)]
http: implement data_sent write callback

Signed-off-by: Luka Perkov <luka@openwrt.org>
7 years agohttp: rework authentication handling - only retry internally for GET requests
Felix Fietkau [Wed, 28 May 2014 09:43:56 +0000 (11:43 +0200)]
http: rework authentication handling - only retry internally for GET requests

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
7 years agouclient-fetch: define _GNU_SOURCE (used for asprintf)
Felix Fietkau [Fri, 9 May 2014 00:16:51 +0000 (02:16 +0200)]
uclient-fetch: define _GNU_SOURCE (used for asprintf)

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
7 years agofetch: print the URL while connecting
Felix Fietkau [Thu, 8 May 2014 21:20:12 +0000 (23:20 +0200)]
fetch: print the URL while connecting

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
7 years agofetch: add support for --user and --password
Felix Fietkau [Thu, 8 May 2014 16:51:16 +0000 (18:51 +0200)]
fetch: add support for --user and --password

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
7 years agofetch: always use O_EXCL when creating output files, use unlink if -O is specified
Felix Fietkau [Thu, 8 May 2014 15:39:51 +0000 (17:39 +0200)]
fetch: always use O_EXCL when creating output files, use unlink if -O is specified

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
7 years agofetch: print download status
Felix Fietkau [Wed, 7 May 2014 23:26:38 +0000 (01:26 +0200)]
fetch: print download status

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
7 years agofetch: indicate an error if the connection was terminated prematurely
Felix Fietkau [Wed, 7 May 2014 21:21:52 +0000 (23:21 +0200)]
fetch: indicate an error if the connection was terminated prematurely

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
7 years agoadd uclient_get_url_filename helper function
Felix Fietkau [Tue, 6 May 2014 13:03:21 +0000 (15:03 +0200)]
add uclient_get_url_filename helper function

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
7 years agofetch: print error messages on http error codes
Felix Fietkau [Tue, 6 May 2014 12:47:38 +0000 (14:47 +0200)]
fetch: print error messages on http error codes

Signed-off-by: Felix Fietkau <nbd@openwrt.org>