Julia Programming - Networking

Sockets and Servers

To deal with streaming I/O objects such as pipes, TCP sockets, terminals, etc., we need a rich interface which is provided by Julia. This Julia interface is provided to the programmer in synchronous manner despite the fact that it is presented asynchronously at system level.

The advantage is that the programmer does not need to think about the underlying asynchronous operations. Before getting deep into this, we should know the concept of well-known ports.

Well-known ports

The concept of well-known ports and networked services on them was introduced in early 1980s by Berkeley development. It was first introduced to Unix. The basic idea behind this was −

  • A particular network service should associate with a particular port number.

  • And the network packet should be sent tagged with that port number.

Some of the well-known ports are as follows −

  • Port 21-file transfer protocol

  • Port 22-SSH

  • Port 25-sendmail

  • Port 80-web servers to deliver HTTP content

  • Port 3306-used commonly by MySQL database

  • Port 28017-used commonly by MongoDB Server

  • Port 6379- Stores Redis key-value

Julia’s UDP and TCP sockets

The internet protocol (IP) specified following two types of sockets −

Unreliable

The concept of unreliable socket rests in the fact that some requests which if not serviced, will be ignored, and retired. Example, requesting the network time from NNTP server. All these kinds of sockets are connectionless and operating via UDP (User Datagram Protocol).

Reliable

The concept of reliable sockets is opposite to unreliable sockets. They are connection full and operate via TCP (Transmission Control Protocol).

Julia supports both these sockets (UDP and TCP) and the source code is provided in socket.jl and streams.jl base modules.

Example

In the example given below, we will be creating a simple server involving TCP sockets −

julia> using Sockets

julia> @async begin
                  server = listen(ip"127.0.0.1",2000)
                  while true
                     sock = accept(server)
                     println("This is TCP server example\n")
                  end
               end
julia> connect(2000)
This is TCP server example

Named Pipes

Named pipes or UNIX domain sockets is a FIFO(First-in, First-out) stream and an extension to the traditional pipe mechanism on Unix and OS X. It is also available on Windows and has a specific pattern for the name prefix (\\.\pipe\). It is a communication channel which uses a special file.

Example

We can also create a named pipe server as given below −

julia> using Sockets

julia> @async begin
                  server = listen("\\\\.\\pipe\\testsocket")
                  while true
                     sock = accept(server)
                     println("This is a named pipe server example\n")
                  end
               end
               
julia> connect(2000)
This is a named pipe server example

A TCP web service

The functionality of a web browser is different from that of an echo server (which we developed earlier in this section). One important difference is that the web server should be able to return different file formats (JPEG, PNG, GIFs, TXT, etc.) and the browser should be able to distinguish between them.

Example

The following example will return a random quote as plain text from a text file −

julia> function web_server(sock::Integer)
               foo = open("/Users/Leekha/Desktop/Hello.txt");
                     header = """HTTP/1.1 200 OK
               Content-type: text/plain; charset=us-ascii
               """ ;
               wb = readlines(foo);
               close(foo);
               wn = length(wb);
               @async begin
               server = listen(sock)
               while true
               wi = rand(1:wn)
               ws = chomp(wb[wi])
               sock = accept(server)
               println(header*ws)
               end
               end
               end
web_server (generic function with 1 method)

julia> web_server(8080)
Task (runnable) @0x0000000014bae570

julia> conn = connect(8080)
HTTP/1.1 200 OK
Content-type: text/plain; charset=us-ascii
Hello, This is Tutorialspoint

TCPSocket(Base.Libc.WindowsRawSocket(0x00000000000003f8) open, 0 bytes waiting)

The Julia Web Group

The web browsers are mainly built with the property to respond to the request issued for a browser. Here we will discuss how we can interact with the Web through HTTP requests (for getting as well as posting data to the web).

First, we need to import the Requests.jl package as follows −

Pkg.add(“Requests”)

Next, import the necessary modules namely get and post as follows −

import Requests: get, post

Use GET request to request data from a specified web browser as follows −

get(“url of the website”)

If you want to request from a specified web page inside the website, use the query parameter as follows −

get(“url of the website”; query = Dict(“title”=>”pagenumber/page name”))

We can also set the timeouts for the GET request as follows −

get(“url of the website”; timeout = 0.2)

We can use the below command to avoid getting your request repeatedly redirected to different websites −

get(“url of the website”; max_redirects = 2)

Using the below command prevents the site from redirecting your GET request −

get(“url of tcommand he website”; allow_redirects = false)

To send the post request, we have to use the below command −

post(“url of the website”)

Using the below command, we can send data to web browser through the POST request −

post(“url of the website”, data = “Data to be sent”)

Let us see how we can send data such as session cookies to web browser through the POST request −

post(“url of the website”, cookies = Dict(“sessionkey”=> “key”)

Files can also be sent as follows −

file = "book.jl"
post("url of the website"; files = [FileParam(file), "text/julia",
"file_name", "file_name.jl"])

WebSockets

We are familiar with the method called AJAX (Asynchronous JavaScript and XML). The example for this method can be the process where we type in the search box and the server returns a set of suggestions and they change as the search term is refined. With this, it is clear that the overhead usage of HTTP protocols is very high.

Web Sockets, which combine the parts of UDP and TCP, are the way to overcome this problem. Hence, web sockets are message-based such as UDP and reliable as TCP. It uses normal HTTP/HTTPS ports, i.e., port 80 and port 443 respectively. They are ideal for vehicles for chat services. Julia provides a package named websockets.jl.

Messaging

Julia supports the following messaging methods −

E-mail

Email is one of the oldest messaging methods. Email messaging can be done in two ways −

  • Sending e-mails − It happens on well-known port 25. Some other ports such as 465 and 587 can also be used for this purpose. SMTP (Simple Mail Transport Protocol) that consists of formulating a message the SMTP server can understand is used for sending emails. To:, From:, and Subject:, all together with the message should be deposited in the mail service’s outbound queue.

  • Receiving e-mails − It is a bit different from sending emails. It basically depends on POP (Post Office Protocol) or IMAP (Internet Message Access Protocol).

Example

Following code can be used to send emails −

using SMTPClient

opt = SendOptions(
   isSSL = true,
   username = "g*****@gmail.com",
   passwd = "yourgmailpassword")
body = IOBuffer(
   "Date: Fri, 25 Sep 2020 19:44:35 +0100\r\n" *
   "From: You <you@gmail.com>\r\n" *
   "To: me@test.com\r\n" *
   "Subject: Test_email\r\n" *
   "\r\n" *
   "Test Message\r\n")
url = "smtps://smtp.gmail.com:465"
rcpt = ["<me@gmail.com>", "<foo@gmail.com>"]
from = "<you@gmail.com>"
resp = send(url, rcpt, from, body, opt)

Twitter

Apart from E-mail, there are other systems that send SMS textual information. One of them is Twitter. Julia provides a package named Twitter.jl to work with Twitter API. To work with Twitter on Julia, we need to authenticate. For authentication, we need to first create an application on dev.twitter.com. After setting up the application, we will be able to access our consumer_key, consumer_token, oauth_token, and oauth_secret.

using Twitter

twitterauth("1234567nOtp...",
            "1234sES96S...",
            "45750-Hjas...",
            "Uonhjlmkmj...")

If you want to say hello to all your Twitter followers, use the following code −

post_status_update("Hello")

And if you want to search the tweets containing the hashtag say #TutorialsPoint, the function call will be as follows −

my_tweets = get_search_tweets("#TutorialsPoint")

The Twitter API bydefault will return the 15 most recent tweets containing the above searched hastag.

Suppose if you want to return the most recent 50 tweets, you can pass the “count” as follows −

my_tweets_50 = get_search_tweets("#TutorialsPoint"; options = {"count" => "50"})

DataFrame method can be defined as follows −

df_tweets = DataFrame(my_tweets_50)

Cloud Services

Julia offers the following cloud services −

The AWS.jl Package

The AWS.jl package is a Julia interface for Amazon Web Services. It replaces AWSCore.jl (provided low-level) and AWSSDK.jl (provided high-level) packages. The AWS.jl package −

  • Includes automated code generation to ensure all new AWS services are available.

  • Keeps the existing service up to date.

We can install this package with the following code −

julia> Pkg.add("AWS")

AWS.jl package can be used with both low-level and high-level API requests. Following are the services supported −

  • EC2

  • S3

  • SQS

  • Auto Scaling

AWSEnv

The structure of AWSEnv is as follows −

type AWSEnv
   aws_id::String # AWS Access Key id
   aws_seckey::String # AWS Secret key for signing requests
   aws_token::String # AWS Security Token for temporary credentials
   region::String # region name
   ep_scheme::String # URL scheme: http or https
   ep_host::String # region endpoint (host)
   ep_path::String # region endpoint (path)
   sig_ver::Int # AWS signature version (2 or 4)
   timeout::Float64 # request timeout in seconds, Default is 0.0
   dry_run::Bool # If true, no actual request will be made
   dbg::Bool # print request and raw response to screen
end

Constructors

Following are the constructors in AWS −

AWSEnv(; id=AWS_ID, key=AWS_SECKEY, token=AWS_TOKEN, ec2_creds=false, scheme="https", region=AWS_REGION, ep="", sig_ver=4, timeout=0.0, dr=false, dbg=false)

Here,

  • AWS_ID and AWS_SECKEY both are initialized from env.

  • AWS_TOKEN − It is by default an empty string.

  • ec2_creds − It should be set to true to automatically retrieve temporary security credentials.

  • region − It should be set to one of the AWS region name strings.

  • ep − It can contain both a hostname and a pathname for an AWS endpoint.

  • sig_ver − It is signature version and must be set to 2 or 4.

Binary Dependencies

Following must be installed before using AWS −

  • libz

  • libxm2

The Google Cloud

The GoogleCloud.jl is the module that wraps GCP (Google Clous Platform) APIs with Julia.

Prerequisites

Following are some prerequisites for Google Cloud −

  • Create a Google account if you do not already have.

  • Need to sign in to the GCP console.

  • You need to create a new project. Click on the Project drop-down menu at the top of the page.

  • You need to first get the credentials from your GCP credentials page, that are associated with your project, as a JSON file.

  • Save this json file to your local computer.

Interacting with the API from Julia

First install the GoogleCloud,jl package as follows −

Pkg.add(“GoogleCloud”)
using GoogleCloud

Now we need to load the service account credentials obtained from Google account −

creds = GoogleCredentials(expanduser("put here address of .json file"))

Create a session as follows −

session = GoogleSession(creds, ["devstorage.full_control"])

By using set_session, set the default session of an API −

set_session!(storage, session)

You can list all the buckets in your existing project as shown below −

bkts = storage(:Bucket, :list)
for item in bkts
   display(item)
   println()
end

Now let us create a new bucket named foo_bkt as follows −

storage(:Bucket, :insert; data=Dict(:name => "foo_bkt"))
bkts = storage(:Bucket, :list)
for item in bkts
   display(item)
   println()
end

You can list all the objects that are in foo_bkt −

storage(:Object, :list, "foo_bkt")

You can delete the bucket as follows −

storage(:Bucket, :delete, "foo_bkt")
bkts = storage(:Bucket, :list)
for item in bkts
   display(item)
   println()
end
Advertisements