A cluster is a collection of stand-alone computers connected using some interconnection network. Each node in a cluster could be a workstation, personal computer, or even a multiprocessor system.A node is an autonomous computer that may be engaged in its private activities while at the same time cooperating with other units in the context of some computational task. Each node has its input/output systems and its operating system.When all nodes in a cluster have the same architecture and run the same operating system, the cluster is called homogeneous, otherwise, it is heterogeneous. The interconnection network could be a fast LAN ... Read More

Parallel applications can be designed using the client/server model. A client may divide a big application into several smaller problems that can be processed by multiple servers simultaneously. All the servers compute the solution to their respective problems and send their results to the client.The client assembles the results from each server and outputs the final result to the user. The client acts as the master (supervisor) while the servers act as the slaves (workers) in the master-slave (supervisor-workers) model as shown in the figure. The steps are taken at the client and each server is summarized as follows.Client (Supervisor)Client ... Read More

Sockets are used to provide the capability of making connections from one application running on one machine to another running on a different machine. A socket abstraction consists of the data structure that holds the information needed for communication, and the system calls that manipulate the socket structure. Once a socket is created, it can be used to wait for an incoming connection (passive socket) or can be used to initiate a connection (active socket).A client can establish an active connection to a remote server by creating an instance of a socket. To establish a server connection and bind it ... Read More

Several possible problems can result from the use of certain routing mechanisms in message-passing systems. These include deadlock, livelock, and starvation, which are as follows −Deadlock − When two messages each hold the resources required by the other to move, both messages will be blocked. This is called a deadlock. It is a phenomenon that occurs whenever there exists a cyclic dependency on resources. Management of resources in a network is the responsibility of the flow control mechanism used. Resources must be allocated in a manner that avoids deadlock.Livelock − Livelock describes a situation in which a message keeps going ... Read More

Shared memory systems can be designed using bus-based or switch-based interconnection networks. The simplest network for shared memory systems is the bus. The bus/cache architecture alleviates the requirement for expensive multiport memories and interface circuitry and the need to adopt a message-passing paradigm when developing application software.The bus may get saturated if multiple processors are trying to access the shared memory (via the bus) simultaneously. A typical bus-based design uses caches to solve the bus contention problem. High-speed caches connected to each processor on one side and the bus on the other side mean that local copies of instructions and ... Read More

Benchmark performance refers to the use of a set of integer and floating-point programs (known collectively as a benchmark) that are designed to test different performance aspects of the computing system(s) under test. Benchmark programs should be designed to provide fair and effective comparisons among high-performance computing systems. For a benchmark to be meaningful, it should evaluate faithfully the performance for the intended use of the system.The examples of benchmarks are the Dhrystone and Whetstone benchmarks. These are synthetic (not real) benchmarks intended to measure the performance of real machines.The Dhrystone benchmark addresses integer performance. It consists of 100 statements ... Read More

It is a point-to-point connection to other nodes or devices. All the network nodes are connected. Mesh has n (n-1)/2 physical channels to link n devices. Mesh architecture with wrap-around connections forms a torus. Several routing mechanisms have been used to route messages around meshes. One such routing mechanism is known as dimension-ordering routing. Using this technique, a message is routed in one given dimension at a time, arriving at the proper coordinate in each dimension before proceeding to the next dimension.There are two techniques to transmit data over the Mesh topology that are as follows −RoutingIn routing, the nodes have a routing logic, ... Read More

Cube-connected networks are patterned after the n-cube structure. An n-cube (hypercube of order n) is defined as an undirected graph having 2n vertices labeled 0 to 2n - 1 such that there is an edge between a given pair of vertices if and only if the binary representation of their addresses differs by one and only one bit. In a cube-based multiprocessor system, processing elements are positioned at the vertices of the graph. Edges of the graph represent the point-to-point communication links between processors.Each processor in a 4-cube is connected to four other processors. In an n-cube, each processor has ... Read More

Static (fixed) interconnection networks are characterized by having fixed paths, unidirectional or bidirectional, between processors. Two types of static networks can be identified. These are completely connected networks (CCNs) and limited connection networks (LCNs).Completely Connected NetworksIn a completely connected network (CCN) each node is connected to all other nodes in the network. Completely connected networks guarantee fast delivery of messages from any source node to any destination node (only one link has to be traversed).Since every node is connected to every other node in the network, routing of messages between nodes becomes a straightforward task. Completely connected networks are, however, ... Read More

Blocking NetworksBlocking networks possess the property that in the presence of a currently established interconnection between a pair of input/output, the arrival of a request for a new interconnection between two arbitrary unused input and output may or may not be possible. Examples of blocking networks include Omega, Banyan, Shuffle–Exchange, and Baseline. Consider, for example SEN, as shown in the figure.In the presence of a connection between input 101 and output 011, a connection between input 100 and output 001 is not possible. This is because the connection 101 to 011 uses the upper output of the third switch from ... Read More