12 Laravel security best practices for 2023

https://res.cloudinary.com/benjamin-crozat/image/upload/dpr_auto,f_auto,q_auto,w_auto/v1690645860/dhjbi100_uiarfm.png

Don’t track your .env file

Your .env file contains sensitive information.

Please, don’t track it!

Make sure it’s included in your .gitignore.

Most of the time, data leaks are inside jobs.

A password manager is a better solution for sharing credentials.

If you want your team members to have access to a curated set of sensitive information, use a password manager with a proven track record of rock-solid security.

Keep Laravel up to date

Keeping Laravel up to date allows you to stay in touch with the latest security updates.

Make sure you are running a version of Laravel that is still being patched, and you should be OK.

If you never upgraded a Laravel project, I wrote a guide that will teach you how to do it.

Keep your first and third-party packages up to date

Access to dozens of packages from the official Laravel ecosystem and thousands of community packages is what makes our job easier.

But the more packages you use, the more points of failure you can be subject to.

Regularly running composer update goes a long way toward a more secure codebase.

composer update in action.

Disable debug messages in production

Make sure these two environment variables are correctly set in production.

APP_ENV=production

APP_DEBUG=false

You don’t want to leak information about your app’s architecture or configuration. Usually, debug messages contain a lot of this kind of details.

Don’t send sensitive information to error monitoring tools

Talking about sensitive information in debug messages, we haven’t eradicated them yet.

If you are using an error monitoring tool, you have to them there as well.

PHP 8.2 introduced a new attribute, \SensitiveParameter, that can hide anything from the stack trace (which is sent to error monitoring tools).

function something(

#[\SensitiveParameter]

$top_secret_parameter,

$who_cares_parameter,

) {

throw new \Exception('Whoops, something bad happended!');

}

Restrict parts of your app with policies

Policies in Laravel are like nightclub bouncers preventing people from accessing restricted areas.

Let’s look at a real-world example of using a Policy:

// app/Policies/PostPolicy.php

public function update(User $user, Post $post)

{

return $user->id === $post->user_id;

}

 

// app/Http/Controllers/PostController.php

public function update(Request $request, Post $post)

{

$this->authorize('update', $post);

 

// ...

}

As you can see, they make it really easy to check for permission before allowing a user to do anything.

Make sure to check the documentation, because there’s a lot to learn about Policies.

Protect your forms from cross-site request forgery (CSRF)

You might want to use the @csrf Blade directive in your forms.

<form method="POST" action="">

@csrf

 

<p>

<label for="name">First Name</label>

<input type="text" id="name" name="name" value="" required />

</p>

 

</form>

This directive generates a hidden input field containing a CSRF token automatically included when submitting the form.

This token confirms that the form is being submitted from your application and not by a third party.

The verification is handled by the VerifyCsrfToken middleware that Laravel uses by default for all your web routes.

Learn more about CSRF protection in Laravel’s documentation.

Validate the user’s input

Validation in Laravel is crucial in ensuring your application’s security.

Validation rules are numerous and will help you sanitize the data your users send with ease. Because you know the drill, right? Never trust your users’ input.

use Illuminate\Http\Request;

 

class PostController extends Controller

{

function store(Request $request)

{

$validated = $request->validate([

'user_id' => 'required|exists:users,id',

'title' => 'required|string|min:3|max:255',

'content' => 'required|string|min:3',

'published' => 'sometimes|boolean'

]);

 

Post::create($validated);

 

//

}

}

Learn more about validation in Laravel on the official documentation.

Be careful with uploaded files

As we saw, the user’s input must never be trusted. That also goes for the files they upload. Here are a few recommendations:

  1. Check the file’s MIME type (Laravel has the right validation rules for that).

$request->validate([

'file' => 'required|mimes:gif,jpeg,png,webp',

]);

  1. When possible, don’t make uploaded files publicly accessible (using the local file driver, for instance).

  2. Upload files on another server. If a hacker bypasses your securities, they won’t be able to run unauthorized code and access sensitive information.

  3. Delegate file uploads to a third-party service reputed for its security (meaning they never leaked data).

Encrypt the payload of your jobs

Whenever you dispatch a job, its payload is saved into the database, Redis or whatever else you told Laravel to use using the QUEUE_DRIVER environment variable.

The payload may contain sensitive information that any of your employee may look at and potentially misuse. As I said in the beginning of this article, leaks are often initiated by employees.

Fortunately, Laravel provides a the Illuminate\Contracts\Queue\ShouldBeEncrypted Contract, which will automatically encrypt the payloads. To make sure nobody can unencrypt them, make sure the Laravel’s APP_KEY defined in your .env is unaccessible to anyone besides a few trusted people.

use Illuminate\Contracts\Queue\ShouldQueue;

use Illuminate\Contracts\Queue\ShouldBeEncrypted;

 

class SomeJob implements ShouldQueue, ShouldBeEncrypted

{

//

}

Write tests for security risks

Testing is unfortunately a vast and lesser-known topic among developers.

Automatically testing multiple scenarii for potential security breaches is a great way to make sure they stay closed.

Laracasts provides free testing courses to help you get started. One with PHPUnit, the industry standard, and one with Pest, the best testing framework on this planet that modernizes and simplifies testing in PHP.

Keep your project tested

Do regular security audits

This practice is one of the most efficient and should be mandatory for anyone that is really serious about security. External feedback can be eyes opening.

As you can imagine, doing security audits isn’t free. It might only be worth it for enterprise since it would cost even more to pay for the potential fines! You cannot put a price on maintaining a good reputation and the trust of your users.

Laravel News Links

TailwindCraft: Free and Open-Source Prebuilt UI Components

https://tailwindcraft.com/storage/photos/1/cover.pngMeticulously designed open-source UI components powered by Tailwind CSS. Streamlines web development with versatile prebuilt elements, seamless Tailwind CSS integration, and a vibrant open-source community.Laravel News Links

Flamethrower Tuba

https://theawesomer.com/photos/2023/08/flaming_tuba_t.jpg

Flamethrower Tuba

Link

There’s nothing inherently dangerous about playing a tuba. Sure, you might run out of breath, but that’s about it. YouTuber and maker MasterMilo has created the most dangerous brass instrument we’ve ever seen. His flamethrower tuba is powered by a chainsaw engine and spews a stream of flaming propane out of its bell.

The Awesomer

A Dive Into toRawSql()


reading up on laravel tricks

Fly.io can build and run your Laravel apps globally, including your scheduled tasks. Deploy your Laravel application on Fly.io, you’ll be up and running in minutes!

In the recent past, we were able to dump out the SQL our query builder was generating like so:

$filter = 'wew, dogs';

// Using the `toSql()` helper
DB::table('foo')
  ->select(['id', 'col1', 'col2'])
  ->join('bar', 'foo.bar_id', 'bar.id')
  ->where('foo.some_colum', $filter)
  ->toSql();

// SELECT id, col1, cole2
//     FROM foo
//     INNER JOIN nar on foo.bar_id = bar.id
//     WHERE foo.some_column = ?

This was useful for debugging complicated queries, but note how we didn’t get the value in our WHERE statement!
All we got was a pesky ? – a placeholder for whatever value we’re passing into the query.
The actual value is hidden from us.

“That’s not what I need”, you may have asked yourself. Assuming we aren’t debugging SQL syntax, our query bindings are what we likely care about the most.

Getting the Full Query

New to Laravel 10.15 is the ability to get the full sql query! That’s much more useful for debugging.

$filter = 'wew, dogs';

// Using the `toRawSql()` helper
DB::table('foo')
  ->select(['id', 'col1', 'col2'])
  ->join('bar', 'foo.bar_id', 'bar.id')
  ->where('foo.some_colum', $filter)
  ->toRawSql();

// SELECT "id", "col1", "col2"
//     FROM "foo" 
//     INNER JOIN "bar" ON "foo"."bar_id" = "bar"."id"
//     WHERE "foo"."some_colum" = 'wew, dogs'"

Much better!

The trick to this is that PDO (the core library used to connect to databases) doesn’t just give this to us – we can only get the SQL with the binding placeholders (hence the old toSql() limitation).

So, we need to build the query with our values within the query ourselves! How’s that done?

How It’s Done

This is tricky business, as we’re dealing with user input – any crazy thing a developer (or their users)
might throw into a sql query needs to be properly escaped.

The new toRawSql() helper stuffs the important logic into a method named substituteBindingsIntoRawSql(). Here’s the PR, for reference.

If we dig into that method code a bit, we’ll
see what’s going on!

The first thing the function does is escape all of the values. This lets Laravel print out the query as a string without worrying about mis-aligned quotes or similar issues.

$bindings = array_map(fn ($value) => $this->escape($value), $bindings);

The call to $this->escape() goes down to a database connection object, and deeper into the underlying PDO object. PDO does the work of
actually escaping the query values in a safe way.

You can get a “Connection Refused” error using the toRawSql() method if your database connection isn’t configured or isn’t working.
That’s because the underlying code uses the “connection” object (PDO under the hood) to escape characters within the output.

Following the escaping-of-values, the method goes through the query character by character!

for ($i = 0; $i < strlen($sql); $i++) {
    $char = $sql[$i];
    $nextChar = $sql[$i + 1] ?? null;

    // and so on
}

The major supported databases use different conventions for escape characters. The code here attempts to find escaped characters and ignore them, lest it tries to
substitute something that looks like a query binding character but isn’t. This is the most fraught bit of code in this new feature.

$query = '';


for ($i = 0; $i < strlen($sql); $i++) {
    $char = $sql[$i];
    $nextChar = $sql[$i + 1] ?? null;

    // Single quotes can be escaped as '' according to the SQL standard while
    // MySQL uses \'. Postgres has operators like ?| that must get encoded
    // in PHP like ??|. We should skip over the escaped characters here.
    if (in_array($char.$nextChar, ["\'", "''", '??'])) {
        // We are building the query string back up - We ignore escaped characters
        // and append them to our rebuilt query string. Since we append
        // two characters, we `$i += 1` so the loop skips $nextChar in our `for` loop
        $query .= $char.$nextChar;
        $i += 1;
    } ...
}

The for loop is rebuilding the query string, but with values substituted in for their ? placeholders. The first check here
is looking for certain escape characters. It needs to know the current character AND the next one to know if it’s an escaped character
and therefore should not do any substitutions.

The next part of our conditional is this:

} elseif ($char === "'") { // Starting / leaving string literal...
    $query .= $char;
    $isStringLiteral = ! $isStringLiteral;
}

If we’re opening an unescaped quote, it means we’re at the start (or end) of a string literal. We set a flag for this case, which
is important in our next check.

elseif ($char === '?' && ! $isStringLiteral) { // Substitutable binding...
    $query .= array_shift($bindings) ?? '?';
}

Here’s the magic. If we’re NOT inside of a string literal, AND we’re not finding an escaped character, AND we find a ? character,
then we can assume it’s a query binding to be substituted with the actual value. We take our array of values and shift it – we remove the
first item in that array and append its value to our query string. (The array of values are in the same order of query binding characters – ? – in the query).

Finally, if we just have a regular character that doesn’t have special meaning, we just append it to our query string:

else { // Normal character...
    $query .= $char;
}

Fly.io ❤️ Laravel

Fly your servers close to your users—and marvel at the speed of close proximity. Deploy globally on Fly in minutes!


Deploy your Laravel app!  

Here’s the whole method, as it stands as I write this:

public function substituteBindingsIntoRawSql($sql, $bindings)
{
    $bindings = array_map(fn ($value) => $this->escape($value), $bindings);

    $query = '';

    $isStringLiteral = false;

    for ($i = 0; $i < strlen($sql); $i++) {
        $char = $sql[$i];
        $nextChar = $sql[$i + 1] ?? null;

        // Single quotes can be escaped as '' according to the SQL standard while
        // MySQL uses \'. Postgres has operators like ?| that must get encoded
        // in PHP like ??|. We should skip over the escaped characters here.
        if (in_array($char.$nextChar, ["\'", "''", '??'])) {
            $query .= $char.$nextChar;
            $i += 1;
        } elseif ($char === "'") { // Starting / leaving string literal...
            $query .= $char;
            $isStringLiteral = ! $isStringLiteral;
        } elseif ($char === '?' && ! $isStringLiteral) { // Substitutable binding...
            $query .= array_shift($bindings) ?? '?';
        } else { // Normal character...
            $query .= $char;
        }
    }

    return $query;
}

That’s basically all there is to the story. We want the entire query to be available with our values!
Here are some (light) caveats we have to get this feature:

  1. We need to make a database connection (thanks to using the PDO library for safe escaping)
  2. The above code MAY contain the occasional bug depending on what values are used
  3. Very long (e.g. binary) values in a query would likely return a complete mess of a query string

Those trade-offs seem fine to me!

Laravel News Links

5 Ways to Retrieve the Last Inserted ID in Laravel

https://laracoding.com/wp-content/uploads/2023/07/5-ways-to-retrieve-the-last-inserted-id-in-laravel_1093.png

In Laravel, after inserting data into a database table, you might need to retrieve the last inserted ID after creating the record. This ID is essential for various tasks, like redirecting users to the newly created resource or performing further operations.

This blog post will guide you through several methods to get the last inserted ID in Laravel by using Eloquent models, the DB facade, or direct access to the PDO instance. Let’s explore the various approaches

Method 1: Retrieving Insert ID Using Model::create

One common way to insert data into the database and get the last inserted ID is by using the create method on an Eloquent model. This method not only inserts the data but also automatically fetches the last inserted ID. You can easily access the ID as a property as: $user->id. Consider the following example:

$user = User::create([
    'name' => 'Steve Rogers',
    'email' => 'captain.america@marvel.com',
    'password' => bcrypt('password123'),
]);

$lastInsertedId = $user->id;

Note that when passing an array of properties to the create function, make sure they are defined as fillable, otherwise the values won’t be written into the database. In our example this means we need to ensure the class User properly defines: protected $fillable = ['name', 'email', 'password'];

Method 2: Retrieving Insert ID Using new Model() and save()

Another approach to insert data and retrieve the last inserted ID is by creating a new instance of the model, setting the attributes, and then calling the save method. You can easily access the ID as a property with $user->id. Let’s consider the following example:

$user = new User();
$user->name = 'Natasha Romanoff';
$user->email = 'black.widow@marvel.com';
$user->password = bcrypt('password123');
$user->save();

$lastInsertedId = $user->id;

Method 3: Retrieving Insert ID Using DB Facade insertGetId()

When you need to insert data without using Eloquent models, you can utilize the insertGetId method provided by the DB facade.

$lastInsertedId = DB::table('users')->insertGetId([
    'name' => 'Bruce Banner',
    'email' => 'hulk@marvel.com',
    'password' => bcrypt('password123'),
    'created_at' =>  \Carbon\Carbon::now(),
    'updated_at' => \Carbon\Carbon::now(),
]);

Note that inserting records using this method will not fill your timestamp values automatically, even if they are defined in your Migration. For this reason we’ve included code to generate them manually using Carbon in the code above.

Method 4: Retrieving Insert ID Using Model::insertGetId

If you are inserting data directly through a model and want to retrieve the last inserted ID, you can use the insertGetId method on the model itself.

$lastInsertedId = User::insertGetId([
    'name' => 'Clint Barton',
    'email' => 'hawkeye@marvel.com',
    'password' => 'password123',
    'created_at' =>  \Carbon\Carbon::now(),
    'updated_at' => \Carbon\Carbon::now(),
]);

Note that inserting records using this method will not fill your timestamp values automatically, even if they are defined in your Migration. For this reason we’ve included code to generate them manually using Carbon in the code above.

Method 5: Direct Access to PDO lastInsertId

You can also directly access the PDO instance to retrieve the last inserted ID.

DB::table('users')->insert([
    'name' => 'Peter Parker',
    'email' => 'spiderman@marvel.com',
    'password' => bcrypt('password123'),
    'created_at' =>  \Carbon\Carbon::now(),
    'updated_at' => \Carbon\Carbon::now(),
]);

$lastInsertedId = DB::getPdo()->lastInsertId();

Note that inserting records using this method will not fill your timestamp values automatically, even if they are defined in your Migration. For this reason, we’ve included code to generate them manually using Carbon in the code above.

Conclusion

By following the methods outlined in this blog post, you can easily obtain the last inserted ID using Eloquent models, the DB facade, or direct access to the PDO instance. Choose the method that suits your needs. However, it’s worth noting that method 1 and method 2 are the most commonly used in Laravel applications. Happy coding!

References

Laravel News Links

Automatically generate RSS feeds in a Laravel application

https://leopoletto.com/assets/images/how-to-generate-rss-feeds-in-a-laravel-application.png

One handy way of keeping users up-to-date on your content is creating an RSS feed.
It allows them to sign up using an RSS reader.
The effort to implement this feature is worth considering because
the website will have another content distribution channel.

Spatie, a well-known company by creating hundreds of good packages for Laravel.
One of them is laravel-feed.
Let’s see how it works:

Installation

The first step is to install the package in your Laravel Application:

composer require spatie/laravel-feed

Then you must publish the config file:

php artisan vendor:publish --provider="Spatie\Feed\FeedServiceProvider" --tag="feed-config"

Usage

Let’s break down the possibilities when configuring a feed.

Creating feeds

The config file has a feeds key containing an array in which each item represents a new feed, and the key is the feed name.

Let’s create a feed for our Blog Posts:

app/config/feed.php

return [
    'feeds' => [
        'blog-posts' => [
            //...
        ],
        'another-feed' => [
            //...
        ]   
    ]
];

The key blog-posts is also the name of the feed in which its value contains the configuration as an Array.
You can create more feeds if needed, but for the sake of this article, let’s focus on blog-posts.

That being said, for our model to work,
we need
to implement the interface Spatie\Feed\Feedable.
It has a signature for a public method named toFeedItem
which must return an instance of Spatie\Feed\FeedItem.

Below is an example of how to create a FeedItem object:

app/Models/BlogPost.php

use Illuminate\Database\Eloquent\Model;
use Spatie\Feed\Feedable;
use Spatie\Feed\FeedItem;

class BlogPost extends Model implements Feedable
{
    //...
    public function toFeedItem(): FeedItem
    {
        return FeedItem::create()
            ->id($this->id)
            ->title($this->title)
            ->summary($this->summary)
            ->updated($this->updated_at)
            ->link(route('blog-posts.show', $this->slug))
            ->authorName($this->author->name)
            ->authorEmail($this->author->email);
    }
}

Now we must create a class with a static method which is going to return a collection of App\Models\BlogPost objects:

app/Feed/BlogPostFeed.php

namespace App\Feed;

use App\Models\BlogPost;
use Illuminate\Database\Eloquent\Collection;

class BlogPostFeed
{
    public static function getFeedItems(): Collection
    {
        return BlogPost::all();
    } 
}

Back to our config file, the first key for our feed configuration is items,
which defines where to retrieve the collection of posts.

app/config/feed.php

return [
    'feeds' => [
        'blog-posts' => [
            'items' => [App\Feed\BlogPostFeed::class, 'getFeedItems']
            //...
        ],
    ]
];

Then you have to define the URL:

app/config/feed.php

return [
    'feeds' => [
        'blog-posts' => [
            //'items' => [App\Feed\BlogPostFeed::class, 'getFeedItems'],
            'url' => '/posts', //https://domain.com/posts
            //...
        ],
    ]
];

Register the routes using a macro feeds included in the package:

app/routes/web.php

//...
Route::feeds();  //https://domain.com/posts

If you wish to add a prefix:

app/routes/web.php

//...
Route::feeds('rss'); //https://domain.com/rss/posts

Following, you must add a title, description and language:

app/config/feed.php

return [
    'feeds' => [
        'blog-posts' => [
            //'items' => [App\Feed\BlogPostFeed::class, 'getFeedItems'],
            //'url' => '/posts',
            'title' => 'My feed',
            'description' => 'The description of the feed.',
            'language' => 'en-US',
            //...
        ],
    ]
];

You can also define the format of the feed and the view that will render it.
The acceptable values are RSS, atom, or JSON:

app/config/feed.php

return [
    'feeds' => [
        'blog-posts' => [
            //'items' => [App\Feed\BlogPostFeed::class, 'getFeedItems'],
            //'url' => '/posts',
            //'title' => 'My feed',
            //'description' => 'The description of the feed.',
            //'language' => 'en-US',
            'format' => 'rss',
            'view' => 'feed::rss',
            //...
        ],
    ]
];

There are a few additional options:

 /*
 * The image to display for the feed. For Atom feeds, this is displayed as
 * a banner/logo; for RSS and JSON feeds, it's displayed as an icon.
 * An empty value omits the image attribute from the feed.
 */
'image' => '',

/*
 * The mime type to be used in the <link> tag. Set to an empty string to automatically
 * determine the correct value.
 */
'type' => '',

/*
 * The content type for the feed response. Set to an empty string to automatically
 * determine the correct value.
 */
'contentType' => '',

The final result of the config file should look like below:

app/config/feed.php

return [
    'feeds' => [
        'blog-posts' => [
            'items' => [App\Feed\BlogPostFeed::class, 'getFeedItems'],
            'url' => '/posts',
            'title' => 'My feed',
            'description' => 'The description of the feed.',
            'language' => 'en-US',
            'format' => 'rss',
            'view' => 'feed::rss',
            'image' => '',
            'type' => '',
            'contentType' => '',
        ],
    ]
];

Automatically generate feed links

Feed readers discover a feed looking for a tag in the head section of your HTML documents:

<link rel="alternate" type="application/atom+xml" title="News" href="/rss/posts">

Add this to your <head>:

@include('feed::links')

Alternatively, use the available blade component:

<x-feed-links />

Conclusion

In this article,
you’ve learned how easy it is to add an RSS feed to your website using the laravel-feed package from Spatie.

If you have any comments,
you can share them in the discussion on Twitter.

Laravel News Links

PATH settings for Laravel

https://laravelnews.s3.amazonaws.com/images/path-featured.jpg

For Laravel development, we often find ourselves typing commands like ./vendor/bin/pest to run project-specific commands.

We don’t need to!

To help here, we can update our Mac (or Linux) $PATH variable.

What’s $PATH?

The $PATH variable sets the directories your system looks for when finding commands to run.

For example, we can type which <cmd> to find the path to any given command:

$ which git

/usr/local/bin/git

My system knew to find git in /usr/local/bin because /usr/local/bin is one directory set in my $PATH!

You can echo out your path right now:

# Output the whole path

echo $PATH

 

# For human-readability, split out each

# directory into a new line:

echo "$PATH" | tr ':' '\n'

Relative Directories in PATH

We can edit our $PATH variable to add in whatever directories we want!

One extremely handy trick is to set relative directories in your $PATH variable.

Two examples are adding ./vendor/bin and ./node_modules/.bin:

# In your ~/.zshrc, ~/.bashrc or, ~/.bash_profile or similar

# Each directory is separated by a colon

PATH=./vendor/bin:./node_modules/.bin:$PATH

Here we prepended our two new paths to the existing $PATH variable. Now, no matter what Laravel application we’re cded into, we can run pest and know we’re running ./vendor/bin/pest, phpunit to run ./vendor/bin/phpunit (and the same for any given Node command in ./node_modules/.bin).

We can also set the current directory . in our $PATH (if it’s not already set – it may be):

# In your ~/.zshrc, ~/.bashrc or, ~/.bash_profile or similar

# Each directory is separated by a colon

# Here we also set the current directory in our PATH

PATH=.:./vendor/bin:./node_modules/.bin:$PATH

This way we can type artisan instead of ./artisan or php artisan.

These are the settings I have in place in Chipper CI so users can run pest or phpunit without having to worry about where the command exists in their CI environments.

Notes

Order also matters in $PATH. When a command is being searched for, the earlier directories are searched first. The system will use the first command found – this means you can over-ride a system command by placing it in a directory earlier in $PATH. That’s why we prepend ./vendor/bin and ./node_modules/.bin into $PATH instead of append it.

You can find all locations of a command like this:

$ which -a git

 

git is /usr/local/bin/git

git is /usr/bin/git

git is /usr/local/bin/git

git is /usr/bin/git

Lastly, in all cases here, the commands should have executable permissions to work like this. This is something to keep in mind when creating your own commands, such as a custom bash script.

Laravel News

7 Jupyter Notebook Tips and Tricks to Maximize Your Productivity

https://static1.makeuseofimages.com/wordpress/wp-content/uploads/2023/07/jupyter-notebook-essential-tips-and-tricks-featured-image-1.jpg

Key Takeaways

  • Understanding the difference between command and edit modes is essential for working with Jupyter Notebook. Each mode provides different functionalities and shortcuts.
  • Accessing and using keyboard shortcuts can save you time by avoiding a series of steps for each operation. Make sure you’re in the right mode when executing shortcuts.
  • Jupyter Notebook allows for customization through extensions or manual customization. Use extensions for easier customization or manually customize by creating a CSS file. Restart the notebook for changes to take effect.

Jupyter Notebook is a web-based interactive computing environment that you can use for data analysis and collaborative coding. It allows the integration of code, text, and visualizations into a single document. It has an extensive ecosystem of libraries available for accomplishing different tasks.

It dominates the data science world when it comes to data analysis, data preprocessing, and feature engineering. Here are some essential tips and tricks to help you make the most out of your notebook experience.

1. Difference Between Command Mode and Edit Mode

Understanding the difference between the command and edit modes is one of the fundamental aspects of working with a Jupyter Notebook. This is because each mode provides different functionalities and shortcuts.

The edit mode is indicated by a green border and is the default mode when you select a cell for editing.

In this mode, you can type and edit code within the cell. To enter edit mode, double-click on a cell or press enter when you select one.

The command mode is indicated by a blue cell border. It is also the default mode when you are not actively editing a cell.

In this mode, you can perform notebook-level operations such as creating, deleting, changing, or executing cells. To switch from edit mode to command mode press the ESc key.

2. Accessing and Using the Keyboard Shortcuts

Jupyter Notebooks has a Keyboard shortcuts dialog that helps you view all available shortcuts. To access it make sure you are in command mode. Then press the H key. A pop-up window such as the one below should appear.

Each shortcut has an explanation of what it does next to it. The commands are divided into those that you can use in command mode and edit mode. Make sure you are in the right mode when executing the respective shortcut. Using these shortcuts will help you save a lot of time as you won’t have to follow a series of steps to accomplish each operation.

3. Using Magic Commands

Magic commands provide additional functionalities that you can use for executing tasks. To use them, prefix the command with a % for line magics and two %% for cell-level magic. Instead of memorizing a few, you can access all the available magic commands by using the %lsmagic command.

On a new cell, run the %lsmagic command. This will display all the available magic commands both in the edit and command mode. To understand what each command does, run the command with a postfix question mark to get its documentation. For example, to understand what the %alias magic command does, run %alias?.

Make sure you understand the mode a command runs on before using it.

4. Customizing the Notebook

Jupyter Notebook allows for user customization if you do not like the default look. You can customize it in one of two ways. You can either customize it manually or use extensions. The easier alternative is to use extensions.

To use extensions, run the following command on a new cell. This command will install jupyter-themes, an extension that comes with predefined themes.

 !pip install jupyterthemes

Then proceed to your terminal or CMD to apply configurations. Start by listing the available themes using the code below.

 jt -l

Then use the following command to apply a theme. Replace the theme name with your desired one.

 jt -t <theme_name>

After applying the theme, restart the Jupyter Notebook for the changes to take place. The output of applying the oceans16 theme is as follows:

If you would like to restore the notebook back to default, use the following command.

 jt -r

The command reverts the Jupyter Notebook to its initial default theme.

To manually customize your notebook, follow the following steps.

Go to the directory where you installed Jupyter Notebook. Find the directory with the name .jupyter. Create a new folder inside it and name it custom. Then create a CSS file in the custom directory and name it custom.css. Finally, open the CSS file with an editor and add your CSS customization code.

After adding the code, restart your Jupyter Notebook for the changes to take effect.

5. Collaboration and Sharing

When you are coding you may want to collaborate with other developers. To achieve this in Jupyter Notebook, you can use version control such as Git. To use Git, initialize a Git repository on your project’s root directory. Then add and commit each change you make to the Jupyter Notebook to the Git repository.

Finally, share the repository with the people you want to collaborate with by pushing it to GitHub. This will allow the collaborators to clone the repository hence accessing your Jupyter Notebook files.

Widget and interactive features aid in helping you create dynamic user interfaces within your notebook.

They give you a way to interact and visualize with your data. Jupyter Notebooks support a few widgets by default. To use more widgets you need to install the ipywidgets library using the following command.

 !pip install ipywidgets

After installing, import the widgets module to use its functionalities.

 import ipywidgets as widgets

You now need to create the widget of your choice. For example, to create a slider widget use the following code:

 slider = widgets.IntSlider(min=0, max=100, value=50, description='Slider:')

Then display the slider.

 display(slider) 

The output is as follows:

You can use the slider for user input and selection of a numeric value within a specified range. There are many widgets that the library supports. To list them use the following line of code:

 dir(widgets)

Look for the widget that supports your requirements from the list.

7. Tips for Efficiency and Performance

To improve the efficiency and performance of your notebook, the following tips come in handy:

  • Limit the output and use progress indicators: This will help you avoid cluttering your notebook with excessive output. Use progress indicators to track the progress of the computation. The tqdm library can be useful for this purpose.
  • Minimize cell execution: Execute only the necessary cells to save on resources. You can achieve this by using Run All Above to run the selected cells.
  • Optimize loops and data processing: Use vectorized operations and optimized libraries. Also, avoid unnecessary loops, especially nested loops. They can impact performance. Instead, utilize built-in functions and methods available in data manipulation libraries.
  • Use cached results: If you have time-consuming computations or data loading, consider caching the results to avoid redundant calculations. Use tools like joblib or Pickle for caching.

How to Improve Your Performance as a Data Scientist

In the data science world, there are many tools that can help you increase your throughput. It can be libraries that you can install in your development environment, IDEs tailored for data analysis, or even browser extensions. Strive to research more on the available tools out there as they can help you simplify your work and save you a lot of time.

MakeUseOf