Designing an API can be easy. As long as you approach it in a logical way, and take each part separately, so you do not over complicate the process. For this article I am going to take a theoretical API and walk through how I would design it.

What we are building?

In this series we will be building a Training Log API, what is this? A training log is a simple referencing platform which allows individuals, or organisations, to make a log of what training has been completed as well as marking attendance for training courses.

Possibly one of the first steps for anyone creating an API is figuring out some sort of business logic, what will this system do extactly. These can theen be broken down into user stories/epics/features and go through some level of project management flow to create what will become our sprints.

Lets figure out what we need

As this is a tutorial, I will not be going completely indepth with what this system will do – it will be a trimmed down API to demonstrate a process. We can break down our requirmeents in a simple list at this stage. I like to think in a behavioural way at this stage:

• As a user, I must be able to login and register
• As a user, I must be able to see courses available
• As a user, I must be able to enrol on a course
• As a user, I must be able to attend training sessions as part of a course
• As a user, I must be able to see my progress on a course and my completed courses

These are some pretty basic requirements for our training log API. However, even these simple requiremnts will give us a good insight into how we should approach designing our API.

Stoplight is a fantastic tool for designing your API, and the Models within your API. It has a great user interface, and has the benefit of creating openapi spec deocuments for you. For example:

Which in turn outputs:

title: User type: object description: An entity that can log into our system properties:  id:  type: integer  uuid:  type: string

For this style system a User is always one of the first things that I build, for the purpose of saving reading space I am ommiting none crucial fields from the models.

One we have an entity that is able to access our system we can start to define the connecting models which will in turn define what our API can do.

Breaking down our requirements

Going through our requirements above, it is easy to see how we could break down our resources and start to map relationships between them. Below I will break down the process:

As a user, I must be able to login and register

So a User model will need to be able to access our system – this part is easy enough to understand.

As a user, I must be able to see courses available

A User needs to be able to see available courses. So we will need a Course model and one of the requirements is that we need to see only available ones: so this will want a boolean flag to mark availability which we can apply through the API.

As a user, I must be able to enrol on a course

So we have a User and we have Courses. Now we need to enrol on them. The easiest option here would be to create a many to many relationship through a pivit table in Laravel it would look like:

<?php class User extends Model {  public function courses()  {  return $this->belongsToMany(Course::class);  } } class Course extends Model {  public function users()  {  return $this->belongsToMany(User::class);  } }

This, however, is not what I am going to do. For this I will create a new model called Enrolment which will capture both User and Course but will give us more extensibility than a standard pivot relation.

As a user, I must be able to attend training sessions as part of a course

So each course will want to be broken into sections, which will have an order to them. From this you can also gather that we will want some meta information to tag onto courses and sections. From working in EdTech previously, I know that a course/section will need to have topics that it covers so that an end user can figure out what they will learn on this course and if they want to do it.

To mark attendance, a simple pivot relation will suffice here for now. I do not think this is the time to think about CQRS and Event Sourcing – while it will give great control and insight into the data, it will be over engineering for what we need.

As a user, I must be able to see my progress on a course and my completed courses

So Users want to Enrol onto a Course, which will be tagged with Topics, which will allow them to Attend the many Sections and see progress along the way.

When you look at the feature requirements and the outputted list, there are other areas where improvements could be added – however it is easy to get carried away with yourself and over engineer an API at this point.

So, Stoplight Studio – putting the piecse together

Below you will see some examples (screenshots) and markup which you can use in your projects:

User Model

title: User type: object description: An entity that can log into our system properties:  id:  type: integer  uuid:  type: string

Course Model

title: Course type: object description: An Entity that Users can Enrol onto properties:  id:  type: integer  uuid:  type: string

Enrolment Model

title: Enrolment type: object description: A entity that allows the linking of Users to Courses properties:  id:  type: integer  uuid:  type: string  user:  $ref: ./user.v1.yaml  course:  $ref: ./course.v1.yaml

Section Model

title: Section type: object properties:  id:  type: integer  uuid:  type: string  course:  $ref: ./course.v1.yaml  order:  type: integer description: An entity that defines parts of a Course

Conclusion

Breaking down requirements can take a little bit of extra thought, but using a tool like Stoplight to visualise the information is very handy – and actually makes the task a lot nicer!

In the next part to this series we will be starting some Laravel code going through how we should set up Laravel for an API, and what changes we might want to consider.

Table Of Contents

Source if you want to download the final project

While building ContestKit there was a feature I wanted to allow users to know if the emails that were sent to the winners were delivered successfully. Thankfully this feature is relatively easy to add because of Mailgun’s amazing API. Let’s create a new Laravel application and get started.

Set up your Laravel Application

To get this process started I always use the Laravel installer to start with a fresh install. (I will be using 6.0.1 which is the most recent at the time of this writing) This will get a Laravel application installed, generate your .env file and install your composer dependencies. Once that is installed we’re going to install the Laravel UI package to quickly scaffold the auth views for our app. This will generate our login and registration views.

composer require laravel/ui

Once that is installed run

php artisan ui vue --auth

Set up Mailgun

Next, you’ll need a Mailgun account. If you don’t have one, you can sign up for a free account here that gives you 10,000 free email sends a month. Once you have created your account. Once you have your account all set up, head over here to get your API information. You’re going to need your Private API key and the domain. Once you have that, open your .env and let’s add some information so our application knows to use Mailgun as our mail provider.

MAIL_DRIVER=mailgun MAILGUN_DOMAIN=yoursendingdomain.com MAILGUN_SECRET=private-api-key

Since we’re going to be using Mailgun as our email driver, we need to make sure we satisfy all the driver prerequisites, in this case, we need to install Guzzle Http so run

composer require guzzlehttp/guzzle.

We’re going to need to come back to Mailgun, later on, to set up some webhooks, but until then let’s get our application sending some emails, creating database records corresponding to the emails sent and get the controllers to handle the incoming webhook information from Mailgun.

Set up our database

Next, let’s get our email message model set up. In your console, run the following command.

$ php artisan make:model Message -m

This command will make a model for you in App/Model call Message.php and it will also make a migration for a table called messages. Let’s open our migration and get our DB schema in. The migration should be called ${datatime-stamp}_create_messages_table.php.

<?php use Illuminate\Support\Facades\Schema; use Illuminate\Database\Schema\Blueprint; use Illuminate\Database\Migrations\Migration; class CreateMessagesTable extends Migration { /** * Run the migrations. * * @return void */ public function up() { Schema::create('messages', function (Blueprint $table) { $table->bigIncrements('id'); $table->string('message_id')->index(); $table->morphs('sender'); $table->dateTime('delivered_at')->nullable(); $table->dateTime('failed_at')->nullable(); $table->integer('opened')->default(0); $table->integer('clicked')->default(0); $table->timestamps(); }); } /** * Reverse the migrations. * * @return void */ public function down() { Schema::dropIfExists('messages'); } }

What this table is going to do is track the outbound email message for us. We’re going to use a sender morphTo relationship so that we can attach outbound messages to any model in our app that we want. This will translate into being able to attach the welcome email we send to the user we sent it to. The $table->morphs('sender'); will create two columns for us, a sender_type which will be represented by the model that sent the message, the second will be the sender_id which will represent the id of the Model.

From there we will store the unique id generated for the email message. Mailgun will send this message id when it sends webhook events. We ask Mailgun to notify us of four different events.

• The successful delivery of a message.
• Any clicks on links in the email (if there are any links).
• Every time the email is opened for that email.
• Any permanently failed deliveries.

This failed delivery event is triggered when Mailgun attempts to send the email several but the receiving email service does not accept it. This usually results from a fake/bad email or your email was blocked by the mail service you were sending it to.

Next, let’s add the message relationship to our App\User model. Open up that and add the following.

<?php namespace App; use App\Message; use Illuminate\Contracts\Auth\MustVerifyEmail; use Illuminate\Foundation\Auth\User as Authenticatable; use Illuminate\Notifications\Notifiable; class User extends Authenticatable { use Notifiable; /** * The attributes that are mass assignable. * * @var array */ protected $fillable = [ 'name', 'email', 'password', ]; /** * The attributes that should be hidden for arrays. * * @var array */ protected $hidden = [ 'password', 'remember_token', ]; /** * The attributes that should be cast to native types. * * @var array */ protected $casts = [ 'email_verified_at' => 'datetime', ]; public function message() { return $this->morphOne(Message::class, 'sender'); } }

This will allow us to attach the outbound message on the user.

Next, let’s define the inverse relationship on the App\Message model.

<?php namespace App; use Illuminate\Database\Eloquent\Model; class Message extends Model { protected $guarded = ['id']; protected $dates = [ 'delivered_at', 'failed_at', ]; public function sender() { return $this->morphTo(); } }

Next, there is a great little tip/trick that you can do to help keep your database records for this sender_type column clean. Currently, if you create a record in your database using this relationship you’ll end up with a DB record that looks like this.

| id | message_id | sender_type | sender_id | delivered_at | failed_at | opened | clicked | created_at | updated_at | |----|--------------------------------------------------|-------------|-----------|--------------|-----------|--------|---------|---------------------|---------------------| | 1 | [email protected] | App\User | 1 | | | | | 2019-09-08 16:52:07 | 2019-09-08 16:52:07 |

You can see that the sender_type is the namespace of your User model. This is fine if you never change the namespace of your model, to clean this up, Laravel has a great Relation::morphMap([...]); method to help make the above a little more readable and a lot less coupled to our namespace.

open your AppServiceProvider.php file and in the register() method add the following.

<?php namespace App\Providers; use Illuminate\Database\Eloquent\Relations\Relation; use Illuminate\Support\ServiceProvider; class AppServiceProvider extends ServiceProvider { /** * Register any application services. * * @return void */ public function register() { Relation::morphMap([ 'users' => 'App\User', ]); } /** * Bootstrap any application services. * * @return void */ public function boot() { // } }

By doing the above, when you create a message now, your database record will look like this.

| id | message_id | sender_type | sender_id | delivered_at | failed_at | opened | clicked | created_at | updated_at | |----|--------------------------------------------------|-------------|-----------|--------------|-----------|--------|---------|---------------------|---------------------| | 1 | [email protected] | users | 1 | | | | | 2019-09-08 16:52:07 | 2019-09-08 16:52:07 |

Isn’t that SO MUCH better? If you want to read more about this, check out this post by Joseph Silber.

Set up our Mailgun webhook controller

All right, we’re ready to move on now. Now let’s make a controller for the webhooks. For this I like to namespace things to keep them organized. In your console enter the following.

$ php artisan make:controller Webhooks/MailgunWebhookController

Next, let’s make a middleware for this controller.

$ php artisan make:middleware MailgunWebhookMiddleware

ext, in your routes folder let’s make a webhooks.php file. In here we’re going to add all our webhook routes. I do this to keep things organized and because we want this URL to not have any of the web middleware stack. I also don’t want to have my webhook URL for Mailgun to be prefixed by /api otherwise you could put it in there as well.

In your newly created webhooks.php file add the following route.

Route::group(['namespace' => 'Webhooks',], function () { Route::post('mailgun', 'MailgunWebhookController'); });

If you were to run php artisan route:list you’ll notice the webhook URL you just added doesn’t come up, we’ll need to update our RouteServiceProvider.php to include the routes in this file. So let’s open that up. In the map() function let’s add a method called mapWebhookRoutes()

<?php namespace App\Providers; use Illuminate\Foundation\Support\Providers\RouteServiceProvider as ServiceProvider; use Illuminate\Support\Facades\Route; class RouteServiceProvider extends ServiceProvider { /** * This namespace is applied to your controller routes. * * In addition, it is set as the URL generator's root namespace. * * @var string */ protected $namespace = 'App\Http\Controllers'; /** * Define your route model bindings, pattern filters, etc. * * @return void */ public function boot() { // parent::boot(); } /** * Define the routes for the application. * * @return void */ public function map() { $this->mapApiRoutes(); $this->mapWebRoutes(); $this->mapWebhookRoutes(); } /** * Define the "web" routes for the application. * * These routes all receive session state, CSRF protection, etc. * * @return void */ protected function mapWebRoutes() { Route::middleware('web') ->namespace($this->namespace) ->group(base_path('routes/web.php')); } /** * Define the "api" routes for the application. * * These routes are typically stateless. * * @return void */ protected function mapApiRoutes() { Route::prefix('api') ->middleware('api') ->namespace($this->namespace) ->group(base_path('routes/api.php')); } /** * Define the "webhook" routes for the application. * * These routes are typically stateless. * * @return void */ protected function mapWebhookRoutes() { Route::prefix('webhooks') ->namespace($this->namespace) ->group(base_path('routes/webhooks.php')); } }

Okay now that we have our route, let’s get our controller set up. Open your MailgunWebhookController.php and let’s add an __invoke() method.

<?php namespace App\Http\Controllers\Webhooks; use App\Http\Controllers\Controller; class MailgunWebhookController extends Controller { public function __invoke() { // we'll handel our inbound webhook here. } }

Once you’ve done that let’s test to see if everything is working. if you run php artisan route:list you should see an output like this.

+--------+----------+----------------------------+------------------+------------------------------------------------------------------------+-------------------------------------------------+ | Domain | Method | URI | Name | Action | Middleware | +--------+----------+----------------------------+------------------+------------------------------------------------------------------------+-------------------------------------------------+ | | GET|HEAD | / | | Closure | web | | | POST | _ignition/execute-solution | | Facade\Ignition\Http\Controllers\ExecuteSolutionController | Facade\Ignition\Http\Middleware\IgnitionEnabled | | | GET|HEAD | _ignition/health-check | | Facade\Ignition\Http\Controllers\HealthCheckController | Facade\Ignition\Http\Middleware\IgnitionEnabled | | | GET|HEAD | _ignition/scripts/{script} | | Facade\Ignition\Http\Controllers\ScriptController | Facade\Ignition\Http\Middleware\IgnitionEnabled | | | POST | _ignition/share-report | | Facade\Ignition\Http\Controllers\ShareReportController | Facade\Ignition\Http\Middleware\IgnitionEnabled | | | GET|HEAD | _ignition/styles/{style} | | Facade\Ignition\Http\Controllers\StyleController | Facade\Ignition\Http\Middleware\IgnitionEnabled | | | GET|HEAD | api/user | | Closure | api,auth:api | | | GET|HEAD | home | home | App\Http\Controllers\[email protected] | web,auth | | | GET|HEAD | login | login | App\Http\Controllers\Auth\[email protected] | web,guest | | | POST | login | | App\Http\Controllers\Auth\[email protected] | web,guest | | | POST | logout | logout | App\Http\Controllers\Auth\[email protected] | web | | | POST | password/email | password.email | App\Http\Controllers\Auth\[email protected] | web,guest | | | GET|HEAD | password/reset | password.request | App\Http\Controllers\Auth\[email protected] | web,guest | | | POST | password/reset | password.update | App\Http\Controllers\Auth\[email protected] | web,guest | | | GET|HEAD | password/reset/{token} | password.reset | App\Http\Controllers\Auth\[email protected] | web,guest | | | POST | register | | App\Http\Controllers\Auth\[email protected] | web,guest | | | GET|HEAD | register | register | App\Http\Controllers\Auth\[email protected] | web,guest | | | POST | webhooks/mailgun | | App\Http\Controllers\Webhooks\MailgunWebhookController | | +--------+----------+----------------------------+------------------+------------------------------------------------------------------------+-------------------------------------------------+

Tip if you want to filter the results to a particular route pattern you can run this php artisan route:list --path=webhooks which will return any routes that match that pattern.

+--------+--------+------------------+------+--------------------------------------------------------+------------+ | Domain | Method | URI | Name | Action | Middleware | +--------+--------+------------------+------+--------------------------------------------------------+------------+ | | POST | webhooks/mailgun | | App\Http\Controllers\Webhooks\MailgunWebhookController | | +--------+--------+------------------+------+--------------------------------------------------------+------------+

Okay, If you got that then you’re golden. Next step let’s get an email that we can send. Let’s make a welcome email for users that register for our app. Since Laravel makes it dead simple to add auth login screens. In your console let’s run these commands. The first will create the auth boilerplate for us which will give us the login and registration pages in our app. Next, we will make a Mailable that we can send our users when they register, finally a listener where we will send that welcome email from.

The Welcome email

$ php artisan make:auth $ php artisan make:mail WelcomeEmail --markdown=emails.welcome $ php artisan make:listener SendWelcomeEmail

Once you’ve run that and you visit your app in the browser, you’ll notice that you now have a login and register nav item. Pretty awesome! Okay now open your EventServiceProvider.php file, in there you’ll notice that there are already events mapped out. let’s add our handler to that Registered event.

<?php namespace App\Providers; use App\Listeners\SendWelcomeEmail; use Illuminate\Auth\Events\Registered; use Illuminate\Auth\Listeners\SendEmailVerificationNotification; use Illuminate\Foundation\Support\Providers\EventServiceProvider as ServiceProvider; use Illuminate\Support\Facades\Event; class EventServiceProvider extends ServiceProvider { /** * The event listener mappings for the application. * * @var array */ protected $listen = [ Registered::class => [ SendWelcomeEmail::class, SendEmailVerificationNotification::class, ], ]; /** * Register any events for your application. * * @return void */ public function boot() { parent::boot(); // } }

Okay now that you have that done, every time someone registers in your app, this handler will be triggered. If you’re wondering what the SendEmailVerificationNotification class does, it’s used to send the verification email if you choose to require that in your application. You can learn more about that from here. let’s start sending emails to welcome our users! in the SendWelcomeEmail class add the following.

<?php namespace App\Listeners; use App\Mail\WelcomeEmail; use Illuminate\Auth\Events\Registered; use Illuminate\Contracts\Queue\ShouldQueue; use Illuminate\Queue\InteractsWithQueue; use Illuminate\Support\Facades\Mail; class SendWelcomeEmail { /** * Handle the event. * * @param object $event * @return void */ public function handle(Registered $event) { Mail::to($event->user)->send(new WelcomeEmail($event->user)); } }

So now, if you go ahead and register you should get an email sent to you that looks like this.

This is the default markdown email that ships with Laravel if you see this, things are good. Now that we’re here, let’s start tracking the and saving the outbound messages. To do this we’ll open our App/Mail/WelcomeEmail.php

We’re going to want to add the following.

<?php namespace App\Mail; use App\User; use Illuminate\Bus\Queueable; use Illuminate\Contracts\Queue\ShouldQueue; use Illuminate\Mail\Mailable; use Illuminate\Queue\SerializesModels; class WelcomeEmail extends Mailable { use Queueable, SerializesModels; /** * Create a new message instance. * * @return void */ public function __construct(User $user) { $this->user = $user; } /** * Build the message. * * @return $this */ public function build() { $this->withSwiftMessage(function ($message) { $this->user->message()->create([ 'message_id' => $message->getId(), ]); }); return $this->markdown('emails.welcome'); } }

In the constructor of this class, we’re going to get passed the User that we’re sending this email to, we will be using this to store the outbound Message. The build() code is where we hook into the SwiftMailer to get the message-id and using the message relationship we create the message record in our database. So let’s try and register again and see what happens. Hopefully, you should see a record created in the messages table linked to the user you just registered? If so let’s move on to the webhooks handling.

Handling the incoming webhook

For this, we’ll need to head back into our Mailgun account. From the dashboard select the sending domain that you configured earlier on and then select the webhooks from the sidebar. I am running Laravel Valet locally so I’m going to run valet share and get a URL for my local app to use.

Once you’re done adding the 4 events we will be tracking, and yes there are more if you want to track more you’ll be able to add them with ease. So now that we have this let’s start updating our messages. The first thing we’re going to do is flesh out our middleware for our webhook controller. This middleware will ensure that the inbound HTTP request is in fact from Mailgun by verifying the webhook signature using our API key. Mailgun does refer to this as a webhook secret but if you look, you’ll notice that it’s the same token/string as your API key.

<?php namespace App\Http\Middleware; use Closure; use Illuminate\Http\Response; use Illuminate\Support\Facades\Log; /** * Validate Mailgun Webhooks * @see https://documentation.mailgun.com/user_manual.html#securing-webhooks */ class MailgunWebhook { /** * Handle an incoming request. * * @param \Illuminate\Http\Request $request * @param \Closure $next * @return mixed */ public function handle($request, Closure $next) { if (!$request->isMethod('post')) { abort(Response::HTTP_FORBIDDEN, 'Only POST requests are allowed.'); } if ($this->verify($request)) { return $next($request); } abort(Response::HTTP_FORBIDDEN); } /** * Build the signature from POST data * * @see https://documentation.mailgun.com/user_manual.html#securing-webhooks * @param $request The request object * @return string */ private function buildSignature($request) { return hash_hmac( 'sha256', sprintf('%s%s', $request->input('timestamp'), $request->input('token')), config('services.mailgun.secret') ); } public function verify($request) { $token = $request->input('signature.token'); $timestamp = $request->input('signature.timestamp'); $signature = $request->input('signature.signature'); if (abs(time() - $timestamp) > 15) { return false; } return hash_hmac('sha256', $timestamp . $token, config('services.mailgun.secret')) === $signature; } }

In the above, we take the inbound request and analyze it to ensure it’s authentic before allowing our controller to handle it. You can read more in the Mailgun docs. Below is an example of the payload you can expect from this webhook.

{ “signature”: { "timestamp": "1529006854", "token": "a8ce0edb2dd8301dee6c2405235584e45aa91d1e9f979f3de0", "signature": "d2271d12299f6592d9d44cd9d250f0704e4674c30d79d07c47a66f95ce71cf55" } “event-data”: { "event": "opened", "timestamp": 1529006854.329574, "id": "DACSsAdVSeGpLid7TN03WA", // ... }, "message": { "headers": { "message-id": "[email protected]", } } }

In the verify method you can see the three main things we have here, the timestamp, the token and the signature. What we need to do is concatenate the timestamp and token values, encode the resulting string with the HMAC algorithm (using your API Key as a key and SHA256 digest mode). Then we compare the result to the signature we in get the webhook payload. We also check if the timestamp is not too old which would make it stale. If everything looks good from here, we let this request through and we handle it in our controller.

<?php namespace App\Http\Controllers\Webhooks; use App\Http\Controllers\Controller; use App\Http\Middleware\Webhooks\MailgunWebhook; use App\Message; use Illuminate\Http\Request; class MailgunWebhookController extends Controller { /** * Constructor. */ public function __construct() { $this->middleware(MailgunWebhook::class); } /** * Handles the Stripe Webhook call. * * @param \Illuminate\Http\Request $request * * @return mixed */ public function __invoke(Request $request) { $data = $request->get('event-data'); $message_id = $data['message']['headers']['message-id']; if ($email = Message::whereMessageId($message_id)->first()) { if ($data['event'] === 'opened' || $data['event'] === 'clicked') { $email->increment($data['event']); } if ($data['event'] === 'delivered' || $data['event'] === 'failed') { $email->update(["{$data['event']}_at" => now()]); } } } }

In the construct method, we define the middleware we want to run to protect this controller, and in the __invoke() method we wrote before we will extract the data we need from the request. The message-id that we set in our messages table is found in message -> headers -> message-id this is id to retrieve the message record we want to update. Next we can find the event type (ie, clicked, opened, delivered, failed) in the event-data -> event, so now all we need to do is add some logic for opened or clicked events we’re going to increment a column, otherwise its a delivered or failed message so we’ll update those timestamps to the current UTC of the event. There you have it.

And there you have it, from here you can add messages to other models in your app, say invoices, or any other event that triggers an email and track if your user got it, opened it or clicked etc.

I hope you have found this useful! If you have any comments or things you think can improve this, send me a message or Tweet or something.. Bonus points if it’s a Drake gif.