Jupa Programming - Networking

Sockets and Servers

To deal with streaming I/O objects such as pipes, TCP sockets, terminals, etc., we need a rich interface which is provided by Jupa. This Jupa interface is provided to the programmer in synchronous manner despite the fact that it is presented asynchronously at system level.

The advantage is that the programmer does not need to think about the underlying asynchronous operations. Before getting deep into this, we should know the concept of well-known ports.

Well-known ports

The concept of well-known ports and networked services on them was introduced in early 1980s by Berkeley development. It was first introduced to Unix. The basic idea behind this was −

A particular network service should associate with a particular port number.

And the network packet should be sent tagged with that port number.

Some of the well-known ports are as follows −

Port 21-file transfer protocol

Port 22-SSH

Port 25-sendmail

Port 80-web servers to depver HTTP content

Port 3306-used commonly by MySQL database

Port 28017-used commonly by MongoDB Server

Port 6379- Stores Redis key-value

Jupa’s UDP and TCP sockets

The internet protocol (IP) specified following two types of sockets −

Unrepable

The concept of unrepable socket rests in the fact that some requests which if not serviced, will be ignored, and retired. Example, requesting the network time from NNTP server. All these kinds of sockets are connectionless and operating via UDP (User Datagram Protocol).

Repable

The concept of repable sockets is opposite to unrepable sockets. They are connection full and operate via TCP (Transmission Control Protocol).

Jupa supports both these sockets (UDP and TCP) and the source code is provided in socket.jl and streams.jl base modules.

Example

In the example given below, we will be creating a simple server involving TCP sockets −

jupa> using Sockets

jupa> @async begin
                  server = psten(ip"127.0.0.1",2000)
                  while true
                     sock = accept(server)
                     println("This is TCP server example
")
                  end
               end
jupa> connect(2000)
This is TCP server example

Named Pipes

Named pipes or UNIX domain sockets is a FIFO(First-in, First-out) stream and an extension to the traditional pipe mechanism on Unix and OS X. It is also available on Windows and has a specific pattern for the name prefix (\.pipe). It is a communication channel which uses a special file.

Example

We can also create a named pipe server as given below −

jupa> using Sockets

jupa> @async begin
                  server = psten("\\.\pipe\testsocket")
                  while true
                     sock = accept(server)
                     println("This is a named pipe server example
")
                  end
               end
               
jupa> connect(2000)
This is a named pipe server example

A TCP web service

The functionapty of a web browser is different from that of an echo server (which we developed earper in this section). One important difference is that the web server should be able to return different file formats (JPEG, PNG, GIFs, TXT, etc.) and the browser should be able to distinguish between them.

Example

The following example will return a random quote as plain text from a text file −

jupa> function web_server(sock::Integer)
               foo = open("/Users/Leekha/Desktop/Hello.txt");
                     header = """HTTP/1.1 200 OK
               Content-type: text/plain; charset=us-ascii
               """ ;
               wb = readpnes(foo);
               close(foo);
               wn = length(wb);
               @async begin
               server = psten(sock)
               while true
               wi = rand(1:wn)
               ws = chomp(wb[wi])
               sock = accept(server)
               println(header*ws)
               end
               end
               end
web_server (generic function with 1 method)

jupa> web_server(8080)
Task (runnable) @0x0000000014bae570

jupa> conn = connect(8080)
HTTP/1.1 200 OK
Content-type: text/plain; charset=us-ascii
Hello, This is Tutorialspoint

TCPSocket(Base.Libc.WindowsRawSocket(0x00000000000003f8) open, 0 bytes waiting)

The Jupa Web Group

The web browsers are mainly built with the property to respond to the request issued for a browser. Here we will discuss how we can interact with the Web through HTTP requests (for getting as well as posting data to the web).

First, we need to import the Requests.jl package as follows −

Pkg.add(“Requests”)

Next, import the necessary modules namely get and post as follows −

import Requests: get, post

Use GET request to request data from a specified web browser as follows −

get(“url of the website”)

If you want to request from a specified web page inside the website, use the query parameter as follows −

get(“url of the website”; query = Dict(“title”=>”pagenumber/page name”))

We can also set the timeouts for the GET request as follows −

get(“url of the website”; timeout = 0.2)

We can use the below command to avoid getting your request repeatedly redirected to different websites −

get(“url of the website”; max_redirects = 2)

Using the below command prevents the site from redirecting your GET request −

get(“url of tcommand he website”; allow_redirects = false)

To send the post request, we have to use the below command −

post(“url of the website”)

Using the below command, we can send data to web browser through the POST request −

post(“url of the website”, data = “Data to be sent”)

Let us see how we can send data such as session cookies to web browser through the POST request −

post(“url of the website”, cookies = Dict(“sessionkey”=> “key”)

Files can also be sent as follows −

file = "book.jl"
post("url of the website"; files = [FileParam(file), "text/jupa",
"file_name", "file_name.jl"])

WebSockets

We are famipar with the method called AJAX (Asynchronous JavaScript and XML). The example for this method can be the process where we type in the search box and the server returns a set of suggestions and they change as the search term is refined. With this, it is clear that the overhead usage of HTTP protocols is very high.

Web Sockets, which combine the parts of UDP and TCP, are the way to overcome this problem. Hence, web sockets are message-based such as UDP and repable as TCP. It uses normal HTTP/HTTPS ports, i.e., port 80 and port 443 respectively. They are ideal for vehicles for chat services. Jupa provides a package named websockets.jl.

Messaging

Jupa supports the following messaging methods −

E-mail

Email is one of the oldest messaging methods. Email messaging can be done in two ways −

Sending e-mails − It happens on well-known port 25. Some other ports such as 465 and 587 can also be used for this purpose. SMTP (Simple Mail Transport Protocol) that consists of formulating a message the SMTP server can understand is used for sending emails. To:, From:, and Subject:, all together with the message should be deposited in the mail service’s outbound queue.

Receiving e-mails − It is a bit different from sending emails. It basically depends on POP (Post Office Protocol) or IMAP (Internet Message Access Protocol).

Example

Following code can be used to send emails −

using SMTPCpent

opt = SendOptions(
   isSSL = true,
   username = "g*****@gmail.com",
   passwd = "yourgmailpassword")
body = IOBuffer(
   "Date: Fri, 25 Sep 2020 19:44:35 +0100
" *
   "From: You <you@gmail.com>
" *
   "To: me@test.com
" *
   "Subject: Test_email
" *
   "
" *
   "Test Message
")
url = "smtps://smtp.gmail.com:465"
rcpt = ["<me@gmail.com>", "<foo@gmail.com>"]
from = "<you@gmail.com>"
resp = send(url, rcpt, from, body, opt)

Twitter

Apart from E-mail, there are other systems that send SMS textual information. One of them is Twitter. Jupa provides a package named Twitter.jl to work with Twitter API. To work with Twitter on Jupa, we need to authenticate. For authentication, we need to first create an apppcation on dev.twitter.com. After setting up the apppcation, we will be able to access our consumer_key, consumer_token, oauth_token, and oauth_secret.

using Twitter

twitterauth("1234567nOtp...",
            "1234sES96S...",
            "45750-Hjas...",
            "Uonhjlmkmj...")

If you want to say hello to all your Twitter followers, use the following code −

post_status_update("Hello")

And if you want to search the tweets containing the hashtag say #TutorialsPoint, the function call will be as follows −

my_tweets = get_search_tweets("#TutorialsPoint")

The Twitter API bydefault will return the 15 most recent tweets containing the above searched hastag.

Suppose if you want to return the most recent 50 tweets, you can pass the “count” as follows −

my_tweets_50 = get_search_tweets("#TutorialsPoint"; options = {"count" => "50"})

DataFrame method can be defined as follows −

df_tweets = DataFrame(my_tweets_50)

Cloud Services

Jupa offers the following cloud services −

The AWS.jl Package

The AWS.jl package is a Jupa interface for Amazon Web Services. It replaces AWSCore.jl (provided low-level) and AWSSDK.jl (provided high-level) packages. The AWS.jl package −

Includes automated code generation to ensure all new AWS services are available.

Keeps the existing service up to date.

We can install this package with the following code −

jupa> Pkg.add("AWS")

AWS.jl package can be used with both low-level and high-level API requests. Following are the services supported −

EC2

S3

SQS

Auto Scapng

AWSEnv

The structure of AWSEnv is as follows −

type AWSEnv
   aws_id::String # AWS Access Key id
   aws_seckey::String # AWS Secret key for signing requests
   aws_token::String # AWS Security Token for temporary credentials
   region::String # region name
   ep_scheme::String # URL scheme: http or https
   ep_host::String # region endpoint (host)
   ep_path::String # region endpoint (path)
   sig_ver::Int # AWS signature version (2 or 4)
   timeout::Float64 # request timeout in seconds, Default is 0.0
   dry_run::Bool # If true, no actual request will be made
   dbg::Bool # print request and raw response to screen
end

Constructors

Following are the constructors in AWS −

AWSEnv(; id=AWS_ID, key=AWS_SECKEY, token=AWS_TOKEN, ec2_creds=false, scheme="https", region=AWS_REGION, ep="", sig_ver=4, timeout=0.0, dr=false, dbg=false)

Here,

AWS_ID and AWS_SECKEY both are initiapzed from env.

AWS_TOKEN − It is by default an empty string.

ec2_creds − It should be set to true to automatically retrieve temporary security credentials.

region − It should be set to one of the AWS region name strings.

ep − It can contain both a hostname and a pathname for an AWS endpoint.

sig_ver − It is signature version and must be set to 2 or 4.

Binary Dependencies

Following must be installed before using AWS −

pbz

pbxm2

The Google Cloud

The GoogleCloud.jl is the module that wraps GCP (Google Clous Platform) APIs with Jupa.

Prerequisites

Following are some prerequisites for Google Cloud −

Create a Google account if you do not already have.

Need to sign in to the GCP console.

You need to create a new project. Cpck on the Project drop-down menu at the top of the page.

You need to first get the credentials from your GCP credentials page, that are associated with your project, as a JSON file.

Save this json file to your local computer.

Interacting with the API from Jupa

First install the GoogleCloud,jl package as follows −

Pkg.add(“GoogleCloud”)
using GoogleCloud

Now we need to load the service account credentials obtained from Google account −

creds = GoogleCredentials(expanduser("put here address of .json file"))

Create a session as follows −

session = GoogleSession(creds, ["devstorage.full_control"])

By using set_session, set the default session of an API −

set_session!(storage, session)

You can pst all the buckets in your existing project as shown below −

bkts = storage(:Bucket, :pst)
for item in bkts
   display(item)
   println()
end

Now let us create a new bucket named foo_bkt as follows −

storage(:Bucket, :insert; data=Dict(:name => "foo_bkt"))
bkts = storage(:Bucket, :pst)
for item in bkts
   display(item)
   println()
end

You can pst all the objects that are in foo_bkt −

storage(:Object, :pst, "foo_bkt")

You can delete the bucket as follows −

storage(:Bucket, :delete, "foo_bkt")
bkts = storage(:Bucket, :pst)
for item in bkts
   display(item)
   println()
end