Regex for URLs and files

muay_tb · September 28, 2016, 8:22am

Hi,

I am looking for a regex that will validate a URL and files accessed in a browser.

For example:

http://www.google.co.uk
http://www.google.com
https://www.google.co.uk
https://www.google.com
ftp://
file:///somefile/on/a/server/accessed/from/browser/file.txt

So far I have:

/^(?:(?:https?|ftp|http):\/\/)(?:\S+(?::\S*)?@)?(?:(?!10(?:\.\d{1,3}){3})(?!127(?:\.\d{1,3}){3})(?!169\.254(?:\.\d{1,3}){2})(?!192\.168(?:\.\d{1,3}){2})(?!172\.(?:1[6-9]|2\d|3[0-1])(?:\.\d{1,3}){2})(?:[1-9]\d?|1\d\d|2[01]\d|22[0-3])(?:\.(?:1?\d{1,2}|2[0-4]\d|25[0-5])){2}(?:\.(?:[1-9]\d?|1\d\d|2[0-4]\d|25[0-4]))|(?:(?:[a-z\u00a1-\uffff0-9]+-?)*[a-z\u00a1-\uffff0-9]+)(?:\.(?:[a-z\u00a1-\uffff0-9]+-?)*[a-z\u00a1-\uffff0-9]+)*(?:\.(?:[a-z\u00a1-\uffff]{2,})))(?::\d{2,5})?(?:\/[^\s]*)?$/i

but this only checks for http, https, ftp

How can I get it to also validate files:

file://c:/a/b/c.txt
file:///some/file/on/a/unix/a.txt

Thanks

rbatte1 · September 28, 2016, 10:29am

Hello muay_tb,

I have tidied up your post with CODE tags are removed the URL formatted that hid most of it from view. Please wrap all code, files, input & output/errors in CODE tags. It makes it far easier to read, does not convert text into URLs or email address and preserves spacing for indenting or fixed width data. Highlight the text and press the toll-button with co over de in it.

On your question, perhaps it would be better to explain more simply what you are going to match against. Is the expression to match these specific cases? If so, you might be better with an extended regular expression of
first string|second string|third string|.... ... etc.

Do you have some sample input that we can consider matching against?

I'm still at a loss as to your eventual aim though.

Kind regards,
Robin

drl · September 28, 2016, 10:48am

Hi.

If you are comfortable using perl and associated library modules:

NAME
    Regexp::Common::URI -- provide patterns for URIs.

SYNOPSIS
        use Regexp::Common qw /URI/;

        while (<>) {
            /$RE{URI}{HTTP}/       and  print "Contains an HTTP URI.\n";
        }

DESCRIPTION
    Patterns for the following URIs are supported: fax, file, FTP, gopher,
    HTTP, news, NTTP, pop, prospero, tel, telnet, tv and WAIS. Each is
    documented in the Regexp::Common::URI::scheme, manual page, for the
    appropriate scheme (in lowercase), except for NNTP URIs which are found in
    Regexp::Common::URI::news.

excerpt from perldoc Regexp::Common::URI , q.v

Best wishes ... cheers, drl

muay_tb · September 29, 2016, 4:07am

Hi,

Thanks for your replies.

I am trying to validate URLS entered.

They all will have a protocol such as http://, https://, ftp://.

I also want to ensure that URLs that are files such as file:///somefile/a.txt is also validated.

Examples could be:

http://www.google.com
https://www.google.com
ftp://server.com/file/a.txt
file:///some/path/to/file.txt

Hope this helps?:wall:

drl · September 29, 2016, 3:44pm

Hi.

Here is a sample of perl validation, driven by a shell script:

#!/usr/bin/env bash

# @(#) s1       Demonstrate URI validation with perl module.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
em() { pe "$*" >&2 ; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C perl

FILE=${1-data1}

pl " Sample perl code, callable at command line:"
cat p1

pl " Input data file $FILE:"
cat $FILE

pl " Results:"
./p1 data1

exit 0

producing:

$ ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution        : Debian 8.6 (jessie) 
bash GNU bash 4.3.30
perl 5.20.2

-----
 Sample perl code, callable at command line:
#!/usr/bin/env perl

# @(#) p1       Demonstrate URI validation module.

use strict;
use warnings;
use Regexp::Common qw /URI/;

# Extended pattern to match HTTPS.
my($https) = qr($RE{URI}{HTTP}{-scheme=>qr/https/});

# Convert to lowercase for match, but retain original for display.
my($uri);
while (<>) {
  chomp;
  $uri = $_;
  $_ = lc;
  /$https/ and print "Contains an HTTPS URI: $uri\n";
  /$RE{URI}{HTTP}/ and print "Contains an HTTP URI: $uri\n";
  /$RE{URI}{file}/ and print "Contains a  FILE URI: $uri\n";
  /$RE{URI}{FTP}/ and print "Contains an FTP URI: $uri\n";
}

exit(0);

-----
 Input data file data1:
http://www.google.com
https://www.google.com
http
HTTP
ftp://server.com/file/a.txt
faux://bad
file:///some/path/to/file.txt
phonehome://///something
helloworld://www.unix.com
http://www.unix.com
HTTP://www.unix.com
http://www.unix.com/shell-programming-and-scripting/268589-regex-urls-files.html

-----
 Results:
Contains an HTTP URI: http://www.google.com
Contains an HTTPS URI: https://www.google.com
Contains an FTP URI: ftp://server.com/file/a.txt
Contains a  FILE URI: file:///some/path/to/file.txt
Contains an HTTP URI: http://www.unix.com
Contains an HTTP URI: HTTP://www.unix.com
Contains an HTTP URI: http://www.unix.com/shell-programming-and-scripting/268589-regex-urls-files.html

Some of this is fairly complicated, so if you or friends do not wish to tackle it, perhaps someone will drop in with a better solution.

Best wishes ... cheers, drl