Conexxus                       IFSF

Open Retailing: A collaboration between
IFSF and Conexxus

Data Dictionary       Design Guidelines

Open Retailing FAQs

Our FAQs are broken down into several broad categories:

APIs


An API (or Application Programming Interface) allows applications to talk to each other. An API specifies the requests, conventions, and data formats that should be used.

What is an API? What does it do?

APIs simplify implementation by abstracting the underlying functionality and only exposing data and actions that the developer needs. The details of how the functionality is accomplished is left to the service; the calling application does not need to understand what actions are required "behind the curtain" to accomplish the outcome. For example, an online shopping application can use a payment API to authorize payment by specifying transaction details (e.g., amount). The online shopping application is not exposed to sensitive cardholder data, as it is encapsulated with the payment service.

Why are APIs useful?

REST (Representational State Transfer) is an architectural approach that relies on stateless operations using existing protocols. A RESTful API is an API that uses HTTP requests to GET, PUT, POST and DELETE data.
Interested in learning more? There are many free tutorials on REST APIs. Here are a few to get you started:
What is a REST or RESTful API?

The OpenAPI Specification (OAS), originally known as the Swagger Specification, is a specification for machine-readable interface files for describing, producing, consuming, and visualizing RESTful web services. Swagger UI is an open source project to visually render documentation defined with the OAS specification. Conexxus and IFSF API standards use OAS V3.

What is OAS? What is Swagger? What is Swagger UI?

YAML (YAML Ain't Markup Language) is a human readable superset of JSON. While JSON is already human readable, YAML makes editing easier and allows insertion of comments, which JSON does not. YAML files can be read with appropriate software program (but are not necessarily human friendly) or they can be rendered into a human friendly format using Swagger UI. Conexxus members may view files from Conexxus and Open Retailing in a Swagger renderer on the Conexxus website wherever you see a filename followed by "swagger" (under My Stuff/My Documents or Standards). IFSF and Conexxus members may also view Open Retailing files at docs.openretailing.org
For additonal information on YAML, visit:
What is YAML? How do I read it?

Conexxus and IFSF have published four guidance documents on the following topics (click on each link to download the latest published version): These are living/working documents. If you are a member of Conexxus or IFSF, you will potenitally find proposed versions in the Open Retailing gitlab under the Work in Progress group/API Design Guidelines project. If you'd like to give feedback on the guidelines and are a member, please raise an issue in the work in progress project. If you are not a member and would like to comment/provide feedback on the published versions, please send an email to support@openretailing.org


Are there API Design Guidelines? Where do I find them?

Security has been considered in the design of all Conexxus/IFSF APIs. There are three mechanisms that have been adopted:
  1. OAuth2: This is an industry-standard authorization framework. It keeps user information within the authorization system; there is no need to store credentials at the client. More suitable for Cloud-based systems, but harder to implement at a single store location or site only systems. The actual authorization is performed using a random access token. OAuth2 defines multiple flows supporting different scenarios. For more technical information, refer to the OAuth2 web page: https://oauth.net/2/
  2. Basic Authentication: Built into HTTP protocol, it relies on a username and password credentials. It is considered basic because it was the first authorization and authentication system implemented natively by the browser, and it provides a basic username and password authentication. There is no mechanism to support access control for API level. In Basic Authentication, the user name and password is base 64 encoded and added to the HTTP header "Authorization." For more information, please refer to the Mozilla page: https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication.
  3. API Key: The API Key is a secret shared across two end points. The API Key is transmitted using a HTTP header “X-API-Key” and it can be used in conjunction with OAuth2 or Basic Authentication. The open retailing APIs are built so the API Key can be rotated.
See API Implementation Guide-Security for more details.

I heard there was no security in the Conexxus/IFSF APIs? Is that true?

An endpoint denotes a URI (or URL) at which you can access an API.

What's an endpoint?

A mock server is a test sever that is useful for injecting errors and other situations so the client (calling) code can be tested effectively.

What's a mock server?

Server Sent Events are part of the "HTML5" specification. They allow a client application to subscribe to an event stream from a web (HTTP) server. For our RESTful Web Service APIs, services are hosted on HTTP servers, and SSE is available. A key benefit of SSE is that although the events "stream" from the service host to the client application, it is the client application that instigates the connection, thus eliminating many PCI issues. SSE is a great technology for notifying listeners when a database changes (e.g., a change in a price book, a new transaction added from a real-time feed).

The Open Retailing API Design Guidelines document Design Rules for APIs OAS3.0 gives standard ways to implement an event stream request in an Open Retailing or Conexxus approved Standard API. It also explains how to define JSON objects to travel over the event (SSE) connection and how to implement those definitions in Open API Specification 3.0 (OAS 3.0) files.

For additonal information, see: https://www.html5rocks.com/en/tutorials/eventsource/basics/

What are Server Sent Events (SSE)?

Web sockets are a way to open two-way communication between a client application and a service host. The data to be sent over the connection is not defined – it’s "free format." Common uses for a web socket include streaming videos or sending very large files. For instance, an API might allow a client to request a Web Socket to see a live camera feed, or see a prerecorded video. Another use could be for sending or receiving a complete "price book" or other large artifact that might be clumsier to send over HTTP.

Note: some developers think of web sockets and server sent event (SSE) channels as interchangeable. That’s not really true – the SSE definition provides an interoperable "disciplined" approach to defining events, and web sockets provide the freedom necessary to solve problems that go beyond providing an event stream.

The Open Retailing API Design Guidelines document Design Rules for APIs OAS3.0 gives standard standard ways to implement a web socket request in an Open Retailing or Conexxus approved Standard API.

For additonal information, see: https://www.html5rocks.com/en/tutorials/websockets/basics/

What are web sockets?

Where is...


Video: Where to Find Donated Work Status and Projects


Where do I find work that has been donated to Conexxus and IFSF for consideration as a future standard?

Git/Gitlab


Git is a distributed version control system that uses "repositories" to track all changes made within a project. In other words, a repository is the file that maintains the history of the project. Instances of git run on a server application such as github, gitlab, bitbucket. These applications are all very similar in that they interact with the git repositories in the same fashion, but some have added features on top of the git features such as issue tracking and code review. Conexxus and IFSF use gitlab as the server application and the instances live here: gitlab.conexxus.org and gitlab.openretailing.org Note: You cannot login to directly to gitlab.conexxus.org at this time.

Gitlab is a hosted environment that provides additional features for using Git collaboratively (i.e., as a group), such as notifications, browsing between projects, and issue tracking. Logging in directly is the way to access these features. However, development and commits are not recommended using this interface. Instead, use either of the following access methods: For additional information on git, visit:

What is Git? What's a git repository? What is Gitlab? How do I access/interace to it?

GitLap supports the SSH (Secure Shell) protocol for communications between Git and its servers. An SSH key pair (one private and one public) is used to authenticate users. Your public SSH key must be set in the gitlab server, which will allow access to anyone who has the corresponding private key. You should never give your private key to anyone.

You will need to install Git to generate an SSH key.
Video: How to install Git and Generate SSH Key - Windows



NOTE: ​After installing Git for Windows and generating your public key, you must email your public key to support@conexxus.org for the Conexxus gitlab instance. You can setup your own key directly in the Open Retailing instance or email it to support@openretailing.org and we will set it up for you. You will receive a confirmation email after it has been set up and you can then install and configure a client (command line or gui) of your choice.

What's an SSH Key? Do I need one? How do I create one? What do I do with it?

Creating Multiple SSH Keys: If you are already using SSH keys to connect to another server, you can create a new key pair to connect to Conexxus, and then tell SSH to use those keys for Conexxus. To create a new key, use:

ssh-keygen -b 4096 -f ~/.ssh/conexxus

You will need to send your new public key (found at ~/.ssh/conexxus.pub) to support@conexxus.org (for the conexxus instance of gitlab) and support@openretailing.org (for the openretailing instance of gitlab) to be added to our system.
Then add the following to your ~/.ssh/config file to assign the keys:

Host: gitlab.conexxus.org>
IdentityFile ~/.ssh/conexxus
IdentityOnly yes

Connection Problems: If you receive an error like "Connection Reset By Peer" this typically indicates that the SSH connection to the gitlab servers are being blocked. Contact your IT department and ask them to "Unblock your SSH Connection on Port 22 to gitlab.conexxus.org" and "Unblock your SSH Connection on Port 22 to gitlab.openretailing.org" and this should solve the problem. Alternatively, you could temporarily connect to a hotspot or other network to get around the firewall blocking (assuming this is permitted by your IT department).

What if I already have an SSH Key for my work (or other) server? What if I get a Connection Reset by Peer error?

The Conexxus and OpenRetailing GitLab instances are privately hosted. Therefore, your public gitlab accounts are not authenticated on these servers. Instead an account will be created based on your Conexxus or IFSF logins. Each gitlab instance will have a unique password which is different than your website login. You may choose to use the same password across accounts.

Many accounts are generated automatically. But, if you find you need access to the gitlab instances:
Please send requests for the Conexxus gitlab instance to support@conexxus.org
Please send requests for the OpenRetailing gitlab instance to support@openretailing.org

How do I get access to the gitlab instances? I already have a gitlab account, why do I need a new one for this work?

If you want to use the command line interface, please refer to the Git Manual. Specifically chapter 2 provides a good introduction to the basic commands you will need to use.

How do I use the command line interface for git?


How do I install SourceTree? How do I load my SSH key into SourceTree?

Everything in gitlab is organized into groups and projects. Groups are created based on work stream and user permissions. Projects belong to groups.

Video: How to Find Project in Gitlab


Where do I find project repositories in gitlab?

Within each project repository, there is a readme file. This is a great place to look to see what's how the project is organized and where files can be found.

Video: How to Locate the Readme File for a Project


How do I navigate a repository? Where do I find things I'm looking for?

A clone is a copy of the remote repository that lives on your local machine. Cloning a repository allows you to work with files on your local machine. This prevents your work from affecting others using the same repository until your changes are complete and pushed back to the remote repository.
Video: How to find the SSH URL and clone a repo



What is a clone? How do I clone a repository?

Commit, Push: Git maintains a history of all changes that have occured within a repository. A commit is the basic unit of work that enables the maintenance of that history. When local development work is completed, you commit that work to your local copy of the repository. A push involves publishing one or more of your commits to the remote repostiory. Your commits remain on your local machine and other people using the repository will not see your changes until a push of those commits is complete. All other git artifacts depend in some way on a commit having occurred.

Branch: Each repository starts with a "master" branch. Additonal branches can be created that are separate work areas that are "forked" off of a previous commit. A branch is a moveable pointer to a snapshot of committed files (i.e., the pointer moves as more files are committed to the branch.) The branch remains separate until a merge back into another branch (usually, but not always, master) is completed. Branches are used to isolate development work, perform bug fixes, and create parallel work streams.

Tag: A tag is a pointer to a snapshot of committed files. Unlike branches, tags do not move as additional commits are made into the repository. Tags are used to mark status of a certain commit (e.g., "ready-for-review") or most commonly, you’ll find them on “master” branches as indications of a version release (e.g., V1.0, V2.0).
Version tags can be used in the file “dependencies.txt” to reference specific versions of referenced repositories required (e.g., the data dictionary.)

Merge: Once a branch has been created and all work completed, that work can be merged to the original branch or even some other branch. If you worked in the recommended manner (i.e., making changes first on your local machine), you must push your work to the remote repositiory on the GitLab Host before merging. Most of the time, the changes on the branch being merged are straightforward with regard to the target branch. If changes occur on the target branch that affect the same files changed on the branch to be merged, conflicts may occur. These must be resolved during the merge. In any case, the end result is all the changes are merged on a single branch. Merging is normally, but not always, done on the GitLab Host. In GitLab, merge operations are most often achieved through a “merge request” which provides for notifications and recording of the reasons for the merge.

Video showing how to locate branches and tags in Sourcetree:


What is a commit? A push? A branch? A tag? A merge?

In brief, when an problem is found or a new feature is desired, here are the steps that should occur:
  1. Create Issue
  2. Create Branch
  3. Do Development Work
  4. Create Merge Request
  5. Merge Completed
For a detailed gitflow explanation, see:

What is the recommended git work flow?

Issues are used to request changes. Issues should only be raised on work in progress projects.

Video: Create an issue


Video: Comment on an open issue


What are issues? How do I create an issue? How do I comment on an open issue?

To associate a branch with an issue, start the name of the branch with the issue number, followed by a non-numeric character (e.g., '-').
Video: Create a branch to do development work. Each branch should have an associated issue.



Video: When you done with development on a branch, raise a merge request which triggers peer review and if no problems are found, it is merged back to master (by staff admins).


How do I create a branch to do dev work against an issue? How do I create a merge request?

Video: Setting notifications in gitlab


How do I set notifications in gitlab?

API Data Dictionary


The API Data Dictionary contains uniquely defined objects and elements used in Open Retailing APIs. It is intended that like data is reused across APIs.

The dictionary is an ongoing work in progress that can be accessed by members and can be found at: https://gitlab.openretailing.org/work-in-progress/api-data-dictionary Members are requested to inform us of missing data elements and objects, as well as requests for corrections, additions, improvements, and clarifications of the existing elements and objects. All proposed changes will be reviewed by work groups within Conexxus and IFSF, and potentially be accepted into the Data Dictionary. Changes and additions are reviewed to assure consistency throughout the dictionary and to avoid duplications of the same name being used for the identical element or object.

Member-only released versions can be found here. Be sure to navigate to the tagged version of your choice. Alternatively, Conexxus members may view/download at https://conexxus.org/standards/api-data-dictionary or IFSF members may view it at https://docs.openretailing.org/.

The publically released V1.0 can be found here. Future versions may be released to the public.

What is the API Data Dictionary? Where do I find it?

Traits are objects that don't quite belong in the data dictionary, but they are still useful and can be re-used. For instance, most reports that contain multiple records will require a "start" and a "limit". Look at those as framework objects that will avoid redefining the same data. Traits are located in a separate folder within the api-data-dictionary repo called "traits".

What is a trait?

The utilities directory in the API Data Dictionary repository provides various definitions that are not specific to a single API. For instance, error codes, how to format a "heartbeat" message, and standards for response value are examples of utilities.

What are utilities?

Tools


Currently there are simulators available for: And the live demo: https://tools.openretailing.org/fp/fpui/fpui.html https://tools.openretailing.org/cd/cdui/cdui.html @C/D - What's live demo mean?

What simulators are available? Where do I find them?

How you run a simulator depends on the simulator. If the simulator comes with an executable binary, (e.g., Dispenser Simulator), you can just double click on that file. If the simulator is Node.js based, most likely you will be able to either use it online or through an Express application running on a Docker container.

How do I run a simulator?

Conexxus/IFSF has no specific requirements. Some popular free tools are: Video: How to install Visual Studio Code




What editor and/or integrated develop environment does Conexxus/IFSF require?

Postman is a very popular tool for testing and documenting APIs. If you've developed an API, Postman can create a small test server (mock server) for you. If can also help you create scripts to test it.
See https://www.postman.com/

What is Postman? When would I use it?

Node.js is a JavaScript interpreter/VM. Node.js is required to run some of the tools and simulators. To install it, navigate to https://nodejs.org/en/, download the installer for your platform and run the executable.

Video: Installing Node.js



What is Node.js? Why do I need it? How do I install it?

NPM stands for "Nodejs Package Management" and it is the tool used to install nodejs modules and applications. For instance, to install the swagger-cli, just type the command npm -g install @apidevtools/swagger-cli. In order to run the OpenReatiling.org API validator, you must first install nodejs and npm. Once those tools are installed, run the command "npm install" at the validator’s root directory. The command "npm install" will read the package.json file and retrieve and install all the dependencies.

What is the Nodejs Package Manager (NPM)? Why do I need it?

Express is a web framework for Node.js. With Express, Node.js applications can be self-contained and don't have to rely on an external web server. In additional, Express provides additional middleware functionality such as the ability to handle errors, parse cookies or perform authentication. For additonal information, see https://expressjs.com/

What is Express? Why do I need it?

Docker is a containerization tool. One of the advantages of using the "Cloud" is that it can perform Horizontal Scaling, i.e., if the traffic to an application increases, more instances of that application can be "spawned" to support the extra load. If the usage goes down, instances of the application can be terminated. Before Docker, the tools used to perform horizontal scaling were Virtual Machines (VMs). VMWare, VirtualBox, Linux KVM and other widely deployed virtualization systems work by virtualizing the entire computer. Virtualization of the entire computer is computationally expensive, as every single component of a physical computer device must be virtualized (CPU, HD, Memory, peripherals). With Docker, only the OS is virtualized and for that reason Docker is considered lightweight. A given computer device can handle many more instances of a Docker container vs Virtual Machines.

What is Docker?

In Conexxus and Open Retailing GitLab environments, we are using the "Continuous Integration/Continuous Deployment" (CI/CD) tools provided by GitLab, called "GitLab CI/CD Pipeline Runner". In general, CI/CD tools automate the process of building and deploying an application: download dependencies, build applications, run tests and then deploy required code to a production server. For instance, we can use CI/CD to build the documentation every time the OpenAPI definition file changes.

What is CI/CD?

The API specification validator is an automated tool to ensure that an API conforms to the API Design Guidelines.
Video: API Specification Validator



What is the API Specification Validator? Where is it? How do I use it?


About Conexxus

Conexxus is a non-profit, member-driven technology organization dedicated to the development and implementation of standards, technologies innovation and advocacy for the convenience store and petroleum market. Conexxus membership collaborates on key present and future industry challenges and innovations. Our work efforts improve profitability by reducing the cost of IT ownership and improving the competitiveness of our members.

About IFSF

IFSF is a community dedicated to enabling technology standards for the benefit of retailers of automotive fuel and energy. IFSF's approach is to collaborate with other standards organisations and relevant institutions to achieve common standards for interoperability of forecourt and convenience devices and services. IFSF depends on support from retailers, suppliers and other industry stakeholders who contribute to, and benefit from, shared standards. To date IFSF has set many standards which have been adopted on thousands of forecourts in an ever increasing number of countries world wide.

© Open Retailing (Conexxus and IFSF) 2020. All rights reserved. info@openretailing.org