Blocking and Nonblocking I/O  Blocking - process suspended until I/O completed  Easy to use and understand  Insufficient for some needs  Nonblocking - I/O call returns as much as available  User interface, data copy (buffered I/O)  Implemented via multi-threading  Returns quickly with count of bytes read or written  Asynchronous - process runs while I/O executes  Difficult to use  I/O subsystem signals process when I/O completed Operating System Concepts with Java – 8th Edition 12.40 Silberschatz, Galvin and Gagne ©2009
View full slide show




Blocking and Nonblocking I/O    Blocking - process suspended until I/O completed  Easy to use and understand  Insufficient for some needs Nonblocking - I/O call returns as much as available  User interface, data copy (buffered I/O)  Implemented via multi-threading  Returns quickly with count of bytes read or written Asynchronous - process runs while I/O executes  Difficult to use  I/O subsystem signals process when I/O completed Operating System Concepts – 8th Edition 13.19 Silberschatz, Galvin and Gagne ©2009
View full slide show




Nonblocking and Asynchronous I/O    Blocking - process suspended until I/O completed  Easy to use and understand  Insufficient for some needs Nonblocking - I/O call returns as much as available  User interface, data copy (buffered I/O)  Implemented via multi-threading  Returns quickly with count of bytes read or written  select() to find if data ready then read() or write() to transfer Asynchronous - process runs while I/O executes  Difficult to use  I/O subsystem signals process when I/O completed Operating System Concepts – 9th Edition 13.23 Silberschatz, Galvin and Gagne ©2013
View full slide show




Categories of OS calls • • • • Blocking - process suspended until I/O completed – Easy to use and understand – Insufficient for some needs Non-blocking - I/O call returns as much as available – User interface, data copy (OS buffered I/O) – Implemented via multi-threading – Returns quickly with count of bytes read or written Asynchronous - process runs while I/O executes – Difficult to use – I/O subsystem signals process when I/O completed Direct Application control – ioctl (on UNIX) allows applications to directly control devices – Driver testing
View full slide show




for Loop Behavior #6 int product = 1; int count; for (count = 1; count <= 5; count++) { product = product * count; } /* for count */ The above program fragment behaves identically the same as: /* Program Trace */ int product = 1; /* product = 1 */ int count; /* count is undefined */ count = 1; /* count == 1, product == product *= count; /* count == 1, product == count++; /* count == 2, product == product *= count; /* count == 2, product == count++; /* count == 3, product == product *= count; /* count == 3, product == count++; /* count == 4, product == product *= count; /* count == 4, product == count++; /* count == 5, product == product *= count; /* count == 5, product == count++; /* count == 6, product == for Loop Lesson 1 CS1313 Spring 2019 1 */ 1 */ 1 */ 2 */ 3 */ 6 */ 6 */ 24 */ 24 */ 120 */ 120 */ 29
View full slide show




C:\UMBC\331\java> java.ext.dirs=C:\JDK1.2\JRE\lib\ext java.io.tmpdir=C:\WINDOWS\TEMP\ os.name=Windows 95 java.vendor=Sun Microsystems Inc. java.awt.printerjob=sun.awt.windows.WPrinterJob java.library.path=C:\JDK1.2\BIN;.;C:\WINDOWS\SYSTEM;C:\... java.vm.specification.vendor=Sun Microsystems Inc. sun.io.unicode.encoding=UnicodeLittle file.encoding=Cp1252 java.specification.vendor=Sun Microsystems Inc. user.language=en user.name=nicholas java.vendor.url.bug=http://java.sun.com/cgi-bin/bugreport... java.vm.name=Classic VM java.class.version=46.0 java.vm.specification.name=Java Virtual Machine Specification sun.boot.library.path=C:\JDK1.2\JRE\bin os.version=4.10 java.vm.version=1.2 java.vm.info=build JDK-1.2-V, native threads, symcjit java.compiler=symcjit path.separator=; file.separator=\ user.dir=C:\UMBC\331\java sun.boot.class.path=C:\JDK1.2\JRE\lib\rt.jar;C:\JDK1.2\JR... user.name=nicholas user.home=C:\WINDOWS C:\UMBC\331\java>java envSnoop -- listing properties -java.specification.name=Java Platform API Specification awt.toolkit=sun.awt.windows.WToolkit java.version=1.2 java.awt.graphicsenv=sun.awt.Win32GraphicsEnvironment user.timezone=America/New_York java.specification.version=1.2 java.vm.vendor=Sun Microsystems Inc. user.home=C:\WINDOWS java.vm.specification.version=1.0 os.arch=x86 java.awt.fonts= java.vendor.url=http://java.sun.com/ user.region=US file.encoding.pkg=sun.io java.home=C:\JDK1.2\JRE java.class.path=C:\Program Files\PhotoDeluxe 2.0\Adob... line.separator=
View full slide show




Not All 20 Point Fonts Are Equal 20  A - Can You Read B - Can You Read C - Can You Read D - Can You Read E - Can You Read F - Can You Read G - Can You Read H - Can You Read I - Can You Read 16  J - Can You Read K - Can You Read L - Can You Read M - Can You Read N - Can You Read O - Can You Read P - Can You Read Q - Can You Read R - Can You Read 14  J - Can You Read K - Can You Read L - Can You Read M - Can You Read O - Can You Read P - Can You Read Q - Can You Read R - Can You Read 12  J - Can You Read K - Can You Read L - Can You Read M - Can You Read N - Can You Read O - Can You Read P - Can You Read Q - Can You Read R - Can You Read My Students Tell Me That They Like The Readability Of Ariel Font I never use fonts smaller than 20 point for lecture.
View full slide show




HDFS (Hadoop Distributed File System) is a distr file sys for commodity hdwr. Differences from other distr file sys are few but significant. HDFS is highly fault-tolerant and is designed to be deployed on low-cost hardware. HDFS provides hi thruput access to app data and is suitable for apps that have large data sets. HDFS relaxes a few POSIX requirements to enable streaming access to file system data. HDFS originally was infrastructure for Apache Nutch web search engine project, is part of Apache Hadoop Core http://hadoop.apache.org/core/ 2.1. Hardware Failure Hardware failure is the normal. An HDFS may consist of hundreds or thousands of server machines, each storing part of the file system’s data. There are many components and each component has a non-trivial prob of failure means that some component of HDFS is always non-functional. Detection of faults and quick, automatic recovery from them is core arch goal of HDFS. 2.2. Streaming Data Access Applications that run on HDFS need streaming access to their data sets. They are not general purpose applications that typically run on general purpose file systems. HDFS is designed more for batch processing rather than interactive use by users. The emphasis is on high throughput of data access rather than low latency of data access. POSIX imposes many hard requirements not needed for applications that are targeted for HDFS. POSIX semantics in a few key areas has been traded to increase data throughput rates. 2.3. Large Data Sets Apps on HDFS have large data sets, typically gigabytes to terabytes in size. Thus, HDFS is tuned to support large files. It provides high aggregate data bandwidth and scale to hundreds of nodes in a single cluster. It supports ~10 million files in a single instance. 2.4. Simple Coherency Model: HDFS apps need a write-once-read-many access model for files. A file once created, written, and closed need not be changed. This assumption simplifies data coherency issues and enables high throughput data access. A Map/Reduce application or a web crawler application fits perfectly with this model. There is a plan to support appending-writes to files in future [write once read many at file level] 2.5. “Moving Computation is Cheaper than Moving Data” A computation requested by an application is much more efficient if it is executed near the data it operates on. This is especially true when the size of the data set is huge. This minimizes network congestion and increases the overall throughput of the system. The assumption is that it is often better to migrate the computation closer to where the data is located rather than moving the data to where the app is running. HDFS provides interfaces for applications to move themselves closer to where the data is located. 2.6. Portability Across Heterogeneous Hardware and Software Platforms: HDFS has been designed to be easily portable from one platform to another. This facilitates widespread adoption of HDFS as a platform of choice for a large set of applications. 3. NameNode and DataNodes: HDFS has a master/slave architecture. An HDFS cluster consists of a single NameNode, a master server that manages the file system namespace and regulates access to files by clients. In addition, there are a number of DataNodes, usually one per node in the cluster, which manage storage attached to the nodes that they run on. HDFS exposes a file system namespace and allows user data to be stored in files. Internally, a file is 1 blocks stored in a set of DataNodes. The NameNode executes file system namespace operations like opening, closing, and renaming files and directories. It also determines the mapping of blocks to DataNodes. The DataNodes are responsible for serving read and write requests from the file system’s clients. The DataNodes also perform block creation, deletion, and replication upon instruction The NameNode and DataNode are pieces of software designed to run on commodity machines, typically run GNU/Linux operating system (OS). HDFS is built using the Java language; any machine that supports Java can run the NameNode or the DataNode software. Usage of the highly portable Java language means that HDFS can be deployed on a wide range of machines. A typical deployment has a dedicated machine that runs only the NameNode software. Each of the other machines in the cluster runs one instance of the DataNode software. The architecture does not preclude running multiple DataNodes on the same machine but in a real deployment that is rarely the case. The existence of a single NameNode in a cluster greatly simplifies the architecture of the system. The NameNode is the arbitrator and repository for all HDFS metadata. The system is designed in such a way that user data never flows through the NameNode. 4. The File System Namespace: HDFS supports a traditional hierarchical file organization. A user or an application can create directories and store files inside these directories. The file system namespace hierarchy is similar to most other existing file systems; one can create and remove files, move a file from one directory to another, or rename a file. HDFS does not yet implement user quotas or access permissions. HDFS does not support hard links or soft links. However, the HDFS architecture does not preclude implementing these features. The NameNode maintains the file system namespace. Any change to the file system namespace or its properties is recorded by the NameNode. An application can specify the number of replicas of a file that should be maintained by HDFS. The number of copies of a file is called the replication factor of that file. This info is stored by NameNode. 5. Data Replication: HDFS is designed to reliably store very large files across machines in a large cluster. It stores each file as a sequence of blocks; all blocks in a file except the last block are the same size. The blocks of a file are replicated for fault tolerance. The block size and replication factor are configurable per file. An application can specify the number of replicas of a file. The replication factor can be specified at file creation time and can be changed later. Files in HDFS are write-once and have strictly one writer at any time. The NameNode makes all decisions regarding replication of blocks. It periodically receives a Heartbeat and a Blockreport from each of the DataNodes in the cluster. Receipt of a Heartbeat implies that the DataNode is functioning properly. A Blockreport contains a list of all blocks on a DataNode
View full slide show




Using Interview Results To Construct Personas Step 2: Map interview subjects to behavioral variables User 8 Place each interviewee variable’s range of values. User 7 within each User User 3 Variable 1: Frequency of photo-taking (in photos/month) User 5 0 Variable 2: Frequency of photo- Never viewing Variable 3: User 9 Frequency of photoNever sharing 1 2 3 4 User User 4 10 7 User 2 User Rarely User 8 User 1 Rarely Variable 4: Photo User 8 User 4 CS 321 organization Lesson Six strategy Unorganized Date Only Personas Page 7 User 6 User 9 10 1 User 4 User User 6 Occasionally User 3 User 2 Occasionally User 7 User 2 User 1 Date & Site 5 User 2 6 User 8 User 3 User 1 7 User 9 Frequently User 7 User 5 User 4 8+ Daily User User 10 6 Frequently Daily User User 10 3 User 9 User 6 User 5 Date & Person All 3
View full slide show




Using Interview Results To Construct Personas Step 3: Identify significant behavior patterns Look for sets of subjects who cluster in several Users 1, 4, & 7 variables. User take few User User pictures and are Variable 1: User User User User User 7 10 Frequency of pretty average 9 5 4 1 6 photo-taking in viewing, 0 1 2 3 4 5 6 7 (in sharing, and User photos/mont Users 5&6 organizing. User User User 8 h) share photos 10 User 4 User User User 3 rather heavily Variable 7 2 6 1 and want 2: Rarel Occasionall Frequentl maximum Frequency Never y y Usery organization. of photoUsers 2 & 3 are User User User User 7 viewing User heavy photo 3 5 Variable User User User User 8 10 9 takers, but only 3: 1 2 4 6 average when it Frequency Never Rarel Occasionall Frequentl comes to of photoy y y User viewing, sharing User 7 User Variable 4: sharing, and User User User 2 User 10 Photo CS 321 organizing. 4 1 3 organizatio 8 Lesson Six Personas n strategyUnorganize Date Only Date & Site Date & Page 8 d Person User 8 User 3 2 8 + User 9 Daily Daily User User 9 User 6 5 All 3
View full slide show




Synchronization  Message passing may be either blocking or non-blocking  Blocking is considered synchronous (e.g. RPC/RMI)   Blocking send has the sender block until the message is received  Blocking receive has the receiver block until a message is available Non-blocking is considered asynchronous (e.g. JMS, IBM MQ)  Non-blocking send has the sender send the message and continue  Non-blocking receive has the receiver receive a valid message or null Operating System Concepts with Java – 8th Edition 3.39 Silberschatz, Galvin and Gagne ©2009
View full slide show




Race Condition  count++ could be implemented as register1 = count register1 = register1 + 1 count = register1  count-- could be implemented as register2 = count register2 = register2 - 1 count = register2  Consider this execution interleaving with “count = 5” initially: T0: producer execute register1 = count {register1 = 5} T1: producer execute register1 = register1 + 1 {register1 = 6} T2: consumer execute register2 = count {register2 = 5} T3: consumer execute register2 = register2 - 1 {register2 = 4} T4: producer execute count = register1 {count = 6 } T5: consumer execute count = register2 {count = 4} Operating System Concepts with Java – 8th Edition 6.7 Silberschatz, Galvin and Gagne ©2009
View full slide show




Modeling Convention for Subsystems and Interfaces Represent subsystems as three items in model: 1. <> package; 2. <> class, 3. subsystem interface (class with stereotype <>). Subsystem package provides a container for the elements in the subsystem. The interaction diagrams describe how the subsystem elements collaborate to implement the operations of the interface the subsystem realizes, Note: <> class actually realizes the interface and will orchestrate the implementation of the subsystem operations. ICourseCatalogSystem <> CourseCatalogSystem <> CourseCatalogSystem Different (additional) interfaces would have their own proxy! 11
View full slide show




System Call Implementation  Typically, a number associated with each system call  System-call interface maintains a table indexed according to these numbers  The system call interface invokes intended system call in OS kernel and returns status of the system call and any return values  The caller need know nothing about how the system call is implemented  Just needs to obey API and understand what OS will do as a result call  Most details of OS interface hidden from programmer by API  Managed by run-time support library (set of functions built into libraries included with compiler) Operating System Concepts – 8th Edition 2.15 Silberschatz, Galvin and Gagne ©2009
View full slide show




System Call Implementation  Typically, a number associated with each system call  System-call interface maintains a table indexed according to these numbers  The system call interface invokes intended system call in OS kernel and returns status of the system call and any return values  The caller need know nothing about how the system call is implemented  Just needs to obey API and understand what OS will do as a result call  Most details of OS interface hidden from programmer by API  Managed by run-time support library (set of functions built into libraries included with compiler) Operating System Concepts – 8th Edition 2.15 Silberschatz, Galvin and Gagne ©2009
View full slide show




Describing SubSystem Dependencies - Subsystems  Subsystem Dependencies on a SubSystem <> Client Support <> Server Support Flexible Server • When a subsystem uses some behavior of an element contained by another subsystem or package, a dependency on the external element is needed. • If the element on which subsystem depends is within a different subsystem, the dependency should be on that SubSystem interface, not on the element within the SubSystem, since we are denied entry to the subsystem. • We know the advantages of such a design… • It also gives the designer total freedom in designing the internal 7 behavior of
View full slide show




System Call Implementation  Typically, a number associated with each system call  System-call interface maintains a table indexed according to these numbers  The system call interface invokes intended system call in OS kernel and returns status of the system call and any return values  The caller need know nothing about how the system call is implemented  Just needs to obey API and understand what OS will do as a result call  Most details of OS interface hidden from programmer by API  Managed by run-time support library (set of functions built into libraries included with compiler) Operating System Concepts Essentials – 8th Edition 2.15 Silberschatz, Galvin and Gagne ©2011
View full slide show




Synchronization  Message passing may be either blocking or non-blocking  Blocking is considered synchronous    Blocking send -- the sender is blocked until the message is received  Blocking receive -- the receiver is blocked until a message is available Non-blocking is considered asynchronous  Non-blocking send -- the sender sends the message and continue  Non-blocking receive -- the receiver receives:  A valid message, or  Null message Different combinations possible  If both send and receive are blocking, we have a rendezvous Operating System Concepts Essentials – 2nd Edition 3.44 Silberschatz, Galvin and Gagne ©2013
View full slide show




Synchronization  Message passing may be either blocking or non-blocking  Blocking is considered synchronous    Blocking send -- the sender is blocked until the message is received  Blocking receive -- the receiver is blocked until a message is available Non-blocking is considered asynchronous  Non-blocking send -- the sender sends the message and continue  Non-blocking receive -- the receiver receives:  A valid message, or  Null message Different combinations possible  If both send and receive are blocking, we have a rendezvous Operating System Concepts – 9th Edition 3.44 Silberschatz, Galvin and Gagne ©2013
View full slide show