Managing your computing ecosystem Pt. 3

Managing your computing ecosystem Pt. 3

Overview

The prospect of universal and interoperable management interfaces is closer to reality than ever. Not only is infrastructure converging, but so is the control and management plane. Last time, we discussed Redfish for managing hardware platforms. This time we will talk about Swordfish for managing storage.

Swordfishswordfish

The goal of Swordfish is to provide scalable storage management interfaces. The interfaces are designed to provide efficient, low footprint management for simple direct attached storage with the ability to scale up to provide easy to use management across cooperating enterprise class storage servers in a storage network.

The Swordfish Scalable Storage Management API specification defines extensions to the Redfish API. Thus a Swordfish service is at the same time a Redfish service. These extensions enable simple, scalable, and interoperable management of storage resources, ranging from direct attached to complex enterprise class storage servers. These extensions are collectively named Swordfish and are defined by the Storage Networking Industry Association (SNIA) as open industry standards.

Swordfish extends Redfish in two principal areas. The first is the introduction of the management and configuration based on service levels. The other is the addition of management interfaces for higher level storage resources. The following sections provide more detail on each.

Service based management

Swordfish interfaces allow the client to get what they want without having to know how the implementation produces the results. As an example, a client might want storage protected so that no more than 5 seconds of data is lost in the event of some failure. Instead of specifying implementation details like mirroring, clones, snapshots, or journaling, the interface allows the client to request storage with a recovery point objective of 5 seconds.   The implementation then chooses how to accomplish this requirement.

The basic ideas are borrowed from ITIL (a set of practices for IT service management that focuses on aligning IT services with the needs of business) and are consistent with ISO/IEC 20000.

A Swordfish line of service describes a category of requirements. Each instance of a line of service describes a service requirement within that category. The management service will typically be configured with a small number of supported choices for each line of service. The service may allow an administrator to create new choices if it is able to implement and enforce that choice. To take an example from airlines, you have seating as one line of service with choices of first, business, and steerage. Another line of service could be meals, with choices like regular, vegetarian, and gluten free. Lines of service are meant to be independent from each other. So, in our airline example, we can mix any meal choice with any seating choice.

Swordfish provides three lines of service covering requirements for data storage, (protection, security, and storage), and two lines of service covering requirements for access to data storage, (connectivity and performance).   Swordfish leaves the specification of specific choices within each of these lines of service to management service implementations.

A Swordfish class of service resource describes a service level agreement (SLA). If an SLA is specified for a resource, the service implementation is responsible for assuring that level of service is provided. For that reason, the management service will typically advertise only a small number of SLAs. The service may allow an administrator to create new SLAs if it is able to implement and enforce that agreement.   The requirements of an SLA represented by a class of service resource are defined by a small set of line of service choices.

Swordfish storage

Swordfish starts with Redfish definitions and then extends them. Redfish specifies drive and memory resources from a hardware centric point of view.   Redfish also specifies volumes as block addressable storage composed from drives. Redfish volumes may be encrypted. Swordfish then extends volumes and adds filesystems, file shares, storage pools, storage groups, and a storage service.   (Object stores are intended to be added in the future.)

A storage service provides a focus for management and discovery of the storage resources of a system.  Two principal resources of the storage service are storage pools and storage groups.

A storage pool is a container of data storage capable of providing capacity that conforms to a specified class of service. A storage pool does not support IO to its data storage.  The storage pool acts as a factory to provide storage resources (volumes, file systems, and other storage pools) that have a specified class of service. The capacity of a storage pool may come from multiple sources and are not all required to be of the same type. The storage pool tracks allocated capacity and may provide alerts when space is low.

A storage group is an administrative collection of storage resources (volumes or file shares) that are managed as a group. Typically, the storage group would be associated with one or more client applications. The storage group can be used to specify that all of its resources share the characteristics of a specified class of service. For example a class of service specifying data protection requirements might be applied to all of the resources in the storage group.

One primary purpose of a storage group is to support exposing or hiding all of the volumes associated with a particular application. When exposed, all clients can access the storage in the group via the specified server and client endpoints. The storage group also supports storage (or crash) consistency across the resources in the storage group.

Swordfish extends volumes and adds support for file systems and file shares, including support for both local and remote replication. Each type supports provisioning and monitoring by class of service. The resulting SLA based interface is a significant improvement for clients over the current practice where the client must know the individual configuration requirements of each product in the client’s ecosystem. Each storage service lists the filesystems, endpoints, storage pools, storage groups, drives and volumes that are managed by the storage service.

Recommendations

These three specifications should form the basis for any Restful system management solution.

As a starting point, OData provides a uniform interface suitable for any data service. It is agnostic to the functions of the service, but it supports inspection of an entity data model via an OData conformant metadata document provided by the service. Because of the generic functionality of the Restful style and with the help of inspection of the metadata document, any OData client can have both syntactic and semantic access to most of the functionality of an OData service implementation. OData is recommended as the basis for any Restful service.

Redfish defines an OData data service that provides a number of basic utility functions as well as hardware discovery and basic system management functions. A Redfish implementation can be very light-weight.   All computing systems should implement a Redfish management service. This recommendation runs the gamut from very simple devices in the IOT space up to enterprise class systems.

Finally, Swordfish extends the Redfish service to provide service based storage management. A Swordfish management service is recommended for all systems that provide advanced storage services, whether host based or network based.

Universal, interoperable management based on well-defined, supported standards. It may still seem like an impossible hope to some. Every day, however, we move closer to a more standard, more manageable infrastructure environment.

~George Ericson @GEricson

Managing Your Computing Ecosystem Pt. 2

Managing Your Computing Ecosystem Pt. 2

Overview

We are making strides toward universal and interoperable management interfaces. These are not only interfaces that will interoperate across one vendor or one part of the stack, but management interfaces that can truly integrate your infrastructure management. Last time, we discussed OData, the Rest standardization. This time we will talk about Redfish for managing hardware platforms.

Redfish redfish

Redfish defines a simple and secure, OData conformant data service for managing scalable hardware platforms. Redfish is defined by a set of open industry standard specifications that are developed by the Distributed Management Task Force, Inc. (DMTF).

The initial development was from the point of view of a Baseboard Management Controller (BMC) or equivalent. Redfish management currently covers bare-metal discovery, configuration, monitoring, and management of all common hardware components. It is capable of managing and updating installed software, including for the operating system and for device drivers.

Redfish is not limited to low-level hardware/firmware management. It is also expected to be deployed to manage higher level functionality, including configuration and management of containers and virtual systems.   In collaboration with the IETF, Redfish is also being extended to include management of networks.

The Redfish Scalable Platforms Management API Specification specifies functionality that can be divided into three areas: OData extensions, utility interfaces, and platform management interfaces. These are described briefly in the following sections.

Redfish OData extensions

Redfish requires at least OData v4 and specifies some additional constraints:

  • Use of HTTP v1.1 is required, with support for POST, GET, PATCH, and DELETE operations, including requirements on many HTTP headers
  • JSON representations are required within payloads
  • Several well-known URIs are specified
    • /redfish/v1/ returns the ServiceRoot resource for locating resources
    • /redfish/v1/OData/ returns the OData service document for locating resources
    • /redfish/v1/$metadata returns the OData metadata document for locating the entity data model declarations.

Redfish also extends the OData metamodel with an additional vocabulary for annotating model declarations. The annotations specify information about, or behaviors of the modeled resources.

Redfish utility interfaces

The utility interfaces provide functionality that is useful for any management domain (for example, these interfaces are used by Swordfish for storage management). These interfaces include account, event, log, session, and task management.

The account service manages access to a Redfish service via a manager accounts and roles.

The event service provides the means to specify events and to subscribe to indications when a defined event occurs on a specified set of resources. Each subscription specifies where indications are sent, this can be to a listening service or to an internal resource, (e.g. a log service).

Each log service manages a collection of event records, including size and replacement policies. Resources may have multiple log services for different purposes.

The session service manages sessions and enables creation of an X-Auth-Token representing a session used to access the Redfish service.

The task service manages tasks that represent independent threads of execution known to the redfish service. Typically tasks are spawned as a result of a long running operation.

The update service provides management of firmware and software resources, including the ability to update those resources.

Redfish platform management interfaces

The principal resources managed by a Redfish service are chassis, computer systems and fabrics. Each resource has its current status. Additionally, each type of resource may have references to other resources, properties defining the current state of the resource, and additional actions as necessary.

Each chassis represents a physical or logical container. It may represent a sheet-metal confined space like a rack, sled, shelf, or module. Or, it may represent a logical space like a row, pod, or computer room zone.

Each computer system represents a computing system and its software-visible resources such as memory, processors and other devices that can be accessed from that system. The computer system can be general purpose system or can be a specialized system like a storage server or a switch.

Each fabric represents a collection of zones, switches and related endpoints. A zone is a collection of involved switches and contained endpoints. A switch provides connectivity between a set of endpoints.

All other subsystems are represented as resources that are linked via one or more of these principal resources. These subsystems include: bios, drives, endpoints, fans, memories, PCIe devices, ports, power, sensors, processors and various types of networking interfaces.

Conclusion

Redfish delivers a standardized management interface for hardware resources. While it is beginning with basic functionality like discovery, configuration and monitoring, it will deliver much more. It will extend into both richer services and cover more than physical resources – e.g. virtual systems, containers, and networks. Redfish is built as an OData conformant service, which makes it the second connected part of an integrated management API stack. Next up – Swordfish.

~George Ericson @GEricson

Managing Your Computing Ecosystem

Managing Your Computing Ecosystem

Overview

There is a very real opportunity to take a giant step towards universal and interoperable management interfaces that are defined in terms of what your clients want to achieve. In the process, the industry can evolve away from the current complex, proprietary and product specific interfaces.

You’ve heard this promise before, but it’s never come to pass. What’s different this time? Major players are converging storage and servers. Functionality is commoditizing. Customers are demanding it more than ever.

Three industry-led open standards efforts have come together to collectively provide an easy to use and comprehensive API for managing all of the elements in your computing ecosystem, ranging from simple laptops to geographically distributed data centers.

This API is specified by:

  • the Open Data Protocol (OData) from Oasis
  • the Redfish Scalable Platforms Management API from the DMTF
  • the Swordfish Scalable Storage Management API from the SNIA

One can build a management service that is conformant to the Redfish or Swordfish specifications that provides a comprehensive interface for the discovery of the managed physical infrastructure, as well as for the provisioning, monitoring, and management of the environmental, compute, networking, and storage resources provided by that infrastructure. That management service is an OData conformant data service.

These specifications are evolving and certainly are not complete in all aspects. Nevertheless, they are already sufficient to provide comprehensive management of most features of products in the computing ecosystem.

This post and the following two will provide a short overview of each.

OData odata

The first effort is the definition of the Open Data Protocol (OData). OData v4 specifications are OASIS standards that have also begun the international standardization process with ISO.

Simply asserting that a data service has a Restful API does nothing to assure that it is interoperable with any other data service. More importantly, Rest by itself makes no guarantees that a client of one Restful data service will be able to discover or know how to even navigate around the Restful API presented by some other data service.

OData enables interoperable utilization of Restful data services. Such services allow resources, identified using Uniform Resource Locators (URLs) and defined in an Entity Data Model (EDM), to be published and edited by Web clients using simple HTTP messages.  In addition to Redfish and Swordfish described below, a growing number of applications support OData data services, e.g. Microsoft Azure, SAP NetWeaver, IBM WebSphere, and Salesforce.

The OData Common Schema Definition Language (CSDL) specifies a standard metamodel used to define an Entity Data Model over which an OData service acts. The metamodel defined by CSDL is consistent with common elements of the UML v2.5 metamodel.   This fact enables reliable translation to your programming language of your choice.

OData standardizes the construction of Restful APIs. OData provides standards for navigation between resources, for request and response payloads and for operation syntax. It specifies the discovery of the entity data model for the accessed data service. It also specifies how resources defined by the entity data model can be discovered. While it does not standardize the APIs themselves, OData does standardize how payloads are constructed and a set of query options and many other items that are often different across the many current Restful data services. OData specifications utilize standard HTTP, AtomPub, and JSON. Also, standard URIs are used to address and access resources.

The use of the OData protocol enables a client to access information from a variety of sources including relational databases, servers, storage systems, file systems, content management systems, traditional Web sites, and more.

Ubiquitous use will break down information silos and will enable interoperability between producers and consumers. This will significantly increase the ability to provide new and richer functionality on top of the OData services.

The OData specifications define:

Conclusion

While Rest is a useful architectural style, it is not a “standard” and the variances in Restful APIs to express similar functions means that there is no standard way to interact with different systems. OData is laying the groundwork for interoperable management by standardizing the construction of Restful APIs. Next up – Redfish.

 

~George Ericson @GEricson