Nuffing.com – Page 101 – Just another WordPress site

October 4, 2022

Laravel Database Transactions: 3 Practical Examples

https://laraveldaily.com/storage/116/laravel-database-transactions.png

Database transactions are very useful for multiple database operations, and Laravel has functionality for them. But what would be the practical examples of WHEN you should use them?

In short, transactions are useful for multiple database operations, when you want to make sure that if either of them fails, all of them would be rolled back automatically.

In this article, I will show three typical examples, in Laravel:

Creating a new record with many-to-many related records
Deleting multiple records for a user
Updating summary table after a new record

Let’s get practical.

Example 1. Many-to-Many with Transaction.

Take a look at this typical Controller code:

public function store(StoreUserRequest $request) {
    $user = User::create($request->validated());
    $user->roles()->attach($request->input('roles'));

    return redirect()->route('users.index');
}

As you can see, there’s a new User record, and then multiple roles are attached to the User. But what if something goes wrong in the second sentence?

Let’s imagine that $request->input('roles') is passed not as array but as an invalid string. What happens then?

Laravel Database Transaction Error

And the worst part is not about the error, but the fact that the User record has been actually saved to the database.

In the case of users, it may have a bad consequence of email being already taken, although the registration hasn’t actually been finished, as the users.email field is unique on the database level.

That’s why it’s beneficial to use a Database Transaction here:

use Illuminate\Support\Facades\DB;

// ...

public function store(StoreUserRequest $request) {
    DB::transaction(function() use ($request) {
        $user = User::create($request->validated());
        $user->roles()->attach($request->input('roles'));
    }

    return redirect()->route('users.index');
}

Notice: Keep in mind that you need to pass use ($request) or any other external variable that you need to use inside of the transaction function.

Now, don’t get me wrong: you will still get the same error “Incorrect integer value: ‘abcd’ for column ‘role_id’ at row 1”. But the User creation statement will be rolled back, and you won’t see the user in the database.

Example 2. Deleting Multiple Records for User

Let’s imagine you want to delete the record which has a lot of hasMany/belongsToMany relationships. You need to delete them as well, right? If you haven’t set the cascadeOnDelete() on the DB level in migrations, you need to do it manually.

Something like this:

$profile->avatar->forceDelete();
MediaTag::whereProfileId($profile->id)->delete();
StatusHashtag::whereProfileId($profile->id)->delete();
DirectMessage::whereFromId($profile->id)->delete();
FollowRequest::whereFollowingId($profile->id)
    ->orWhere('follower_id', $profile->id)
    ->forceDelete();
Follower::whereProfileId($profile->id)
    ->orWhere('following_id', $profile->id)
    ->forceDelete();
Like::whereProfileId($profile->id)->forceDelete();

// ... only then delete the profile itself:
$profile->delete();

Imagine what happens if some middle sentence in this code snippet fails. So we have deleted something but not everything?

Of course, compared to the previous example, the consequence isn’t as harsh, because, well, we still want to delete those records anyway, we would just do it later.

But still, the profile would remain active but wouldn’t see some of their data, like username without avatar. Not cool, right?

Just add a few lines of code:

DB::transaction(function() use ($profile) {
    $profile->avatar->forceDelete();
    MediaTag::whereProfileId($profile->id)->delete();
    StatusHashtag::whereProfileId($profile->id)->delete();
    DirectMessage::whereFromId($profile->id)->delete();
    FollowRequest::whereFollowingId($profile->id)
        ->orWhere('follower_id', $profile->id)
        ->forceDelete();
    Follower::whereProfileId($profile->id)
        ->orWhere('following_id', $profile->id)
        ->forceDelete();
    Like::whereProfileId($profile->id)->forceDelete();

    $profile->delete();
});

Example 3. Updating “Summary” Table

Imagine a project with users and financial operations. Usually, they would be called “Transactions”, but to avoid confusion with the subject of the article, I will call them just “Expenses”.

You need to track all Expenses and also the current balance of each user. So, after every purchase, you would do something like this:

Expense::create($expenseDetails);
$user->decrement('balance', $expenseDetails->amount);

Sounds trivial, but in more complex scenarios, you would also need to update some more summary data in other tables, for some reporting.

Here, the consequence of not using DB transactions is huge: users would have more money to spend than they should.

Let’s fix this:

DB::transaction(function() use ($expenseDetails, $user) {
    Expense::create($expenseDetails);
    $user->decrement('balance', $expenseDetails->amount);
});

So, these are just three simple examples of DB Transactions. I hope they will push you towards making your data correct all the time.

Laravel News Links

October 3, 2022

Don’t Use $request->all(): It’s Insecure

https://laraveldaily.com/storage/118/laravel-request-all.png

Quite often, I see Laravel developers using $request->all() in Controller methods. It may be a security issue, let me show you why.

Why $request->all() is insecure

Let’s imagine you have a regular registration form:

Laravel registration form

This form is submitted to this Controller method:

public function store(StoreUserRequest $request) {
    User::create($request->all());

    return redirect()->route('dashboard');
}

We use a Form Request class with validation, so it doesn’t look harmful, does it? It should save name, email, and password, right?

Notice: I know that the password should be encrypted, but in this article, let’s assume the encryption is done somewhere else, like in Observer or Mutator.

Now, let’s take a look at the $fillable array in the User model.

class User extends Authenticatable
{
    protected $fillable = [
        'name',
        'email',
        'password',
        'is_admin',
    ];

See that is_admin column? It is used to assign the administrator role, and that field should be filled only by other administrators, in some other form than the registration, in a separate admin panel.

But what if I try to call that registration to submit by adding a hidden field called is_admin, directly from my browser, like Chrome dev tools, clicking Inspect?

Laravel registration inspect

Laravel registration hidden field

Guess what: the is_admin will be successfully saved, and I will successfully register myself as an administrator, without anyone’s permission.

So, to “hack” the system, all I would need is to guess the non-visual database fields: it may be called is_admin, it may be role_id, just role, or whatever else. Not that hard to write a script to automate trying all the possible options.

This is happening because $request->all() doesn’t filter or validate anything, it’s just literally all().

So, what to do instead?

Option 1. Form Request and validated()

If you use the Form Request class for the validation, you have the rules() method there:

class StoreUserRequest extends FormRequest
{
    public function rules()
    {
        return [
            'name' => ['required', 'string', 'max:255'],
            'email' => ['required', 'email', 'string', 'max:255'],
            'password' => ['nullable', 'string', 'confirmed', 'min:8'],
        ];
    }
}

Then, in the Controller, you should use the request filtered after that validation. For that, just replace $request->all() with $request->validated().

public function store(StoreUserRequest $request) {
    User::create($request->validated());

    return redirect()->route('dashboard');
}

So, it will fill in only the fields that are present in the rules() method.

Keep in mind, that in this case, you need to add all the needed fields into the rules() array, even if it doesn’t require a specific validation, just add them as nullable or string.

Option 2. $request->only()

Of course, another option is to specify the exact fields to be used. One of the options is to use $request->only():

public function store(StoreUserRequest $request) {
    User::create($request->only('name', 'email', 'password'));

    return redirect()->route('dashboard');
}

Option 3. Field by Field

Finally, the good old way of specifying field by field. The code looks longer, but maybe more readable, with less “hidden” information.

public function store(StoreUserRequest $request) {
    User::create([
        'name' => $request->name,
        'email' => $request->email,
        'password' => $request->password,
    ]);

    return redirect()->route('dashboard');
}

All three options above are valid, it’s just your personal preference, which may also depend on the exact form or situation. The main thing is not to use $request->all(), just forget about that method’s existence, for your safety and security.

Laravel News Links

October 2, 2022

A Laravel package to manage dynamic servers

https://freek.dev/og-images/38c00d66ffaaa8eb3dd0f18d49a128c9/2344.png

I’m proud to announce that our team has released a new package called laravel-dynamic-servers.

This package can help start and stop servers when you need them. The prime use case is to spin up extra working servers to help you process the workload on queues.

In this blog post, I’d like to introduce the package to you.

Why we’ve created this package

We are currently beta testing the hosted version of Mailcoach: an easy-to-use, affordable, platform to send email campaigns, and we also support drip campaigns if that’s what you need.

One of the important aspects of Mailcoach is privacy for those who need it. All open- and click tracking is optional. But we go farther from that. For GDPR-sensitive organizations, such as the government, our legal experts suggested that we would only use infrastructure located on EU territory and owned by EU-based companies.

I feel that if you ask 100 legal experts for advice on this, you’ll get 100 different suggestions. It might have been okay to stay on AWS, but we wanted to be on the safe side and avoid any American-owned companies.

The first version of Mailcoach ran on AWS Lambda via Laravel Vapor. We migrated it to UpCloud servers, a European-located and owned hosting company. Those servers are provisioned via Laravel Forge.

The significant benefit running servers on AWS infrastructure gave us was the automatic scaling. We need autoscaling when people simultaneously start sending campaigns to large email lists. During our beta, this scenario already played out: our service had to send out hundreds of thousands of emails (via the queue) in a short period.

UpCloud is not a serverless platform but uses regular servers. A traditional solution to autoscale servers and orchestrating server stuff would be to use Kubernetes. That piece of software is also very powerful but also hard to learn.

That’s why we invested time in creating a package that allows us to spin up extra servers to handle queued jobs whenever there is extra workload and to destroy them whenever the work is done.

Using dynamic servers

You can think of laravel-dynamic-servers as a sort of PHP-based version of Kubernetes that has 5% of its features but covers that 80% use case. For most PHP and Laravel developers, this package will also be easier to learn and use.

The package is driver based. It ships with support for UpCloud (because we needed that ourselves), but the community already created a DigitalOcean driver, and creating a driver of your own is easy.

Typically, on your hosting provider, you would prepare a server snapshot that will be used as a template when starting new servers.

After the package is installed and configured, you can use PHP to start and stop servers.

Here’s the most straightforward way to start a server via PHP code:

use Spatie\DynamicServers\Facades\DynamicServers;

DynamicServers::increase();

To start a server, the package will start a queued job that makes an API call to your server provider to spin up a server. It will also dispatch subsequent jobs to monitor the entire starting process of a server.

Stopping one server is equally simple:

DynamicServers::decrease();

In most cases, you would use these methods directly, though. The package also offers a method called ensure. You can pass it the number of servers you want to have available in total.

DynamicServers::ensure(5);

If fewer servers are currently active than the number given, more servers will spin up. The package will destroy a few if there are more servers than that number.

Usually, you would have code that calculates the number of servers you need and pass that number to ensure.

Here’s a simplified version of the calculation logic we used at Mailcoach. We use Horizon’s WaitTimeCalulator class to get the waiting time of the queue.

use Laravel\Horizon\WaitTimeCalculator;
use Spatie\DynamicServers\Facades\DynamicServers;

$waitTimesOfAllQueues = (WaitTimeCalculator::class)->calculate();

$maxWaitTime = max($waitTimesOfAllQueues);

// 1 server for every 5 minutes of wait time
$amountOfServersNeeded = floor($waitTimesOfAllQueues / 60 / 5); 

DynamicServers::ensure($amountOfServersNeeded);

So, when the wait times are long, the number of needed servers will be passed to ensure. When the queues are empty, $amountOfServersNeeded will be zero. When zero is passed to ensure, all dynamic servers will be destroyed.

Of course, this logic should be executed frequently. The package has a method determineServerCount which will be executed every minute through a scheduled command. You would typically use it in a service provider:

use Laravel\Horizon\WaitTimeCalculator;
use Spatie\DynamicServers\Facades\DynamicServers;
use Spatie\DynamicServers\Support\DynamicServersManager;

// in some service provider

DynamicServers::determineServerCount(function (DynamicServersManager $servers) {
	$waitTimesOfAllQueues = (WaitTimeCalculator::class)->calculate();
	
	$maxWaitTime = max($waitTimesOfAllQueues);
	
	// 1 server for every 5 minutes of wait time
	$amountOfServersNeeded = floor($waitTimesOfAllQueues / 60 / 5); 
	
	DynamicServers::ensure($amountOfServersNeeded);
});

Suppose you don’t want to possibly wait for a minute until the next invocation of the schedule. In that case, you could additionally listen to Horizon’s LongWaitDetected event and immediately execute the logic above to increase the server count.

In closing

The package contains a lot more features that were not mentioned in this blog post, such as using multiple server types, rebooting servers, setting a server limit, and much more.

You can see the source code of laravel-dynamic-servers in this repo on GitHub.

Should you send a large campaign via Mailcoach, you can be sure that a couple of servers will be spun up for you behind the scenes.

A special thanks to my colleague Rias who picked up this idea from Jmac’s recent streams and blog post on spawning workers based on queue workload.

This isn’t the first package that our team has built. On our company website, check out all our open source packages in this long list. If you want to support us, consider picking up any of our paid products.

Laravel News Links

October 1, 2022

You will not believe how perfectly this 1979 Monty Python skit predicted today’s gender madness

https://media.notthebee.com/articles/63399395def0363399395def04.jpg

Just hit play man:

Not the Bee

September 29, 2022

Can’t make it to Dallas? Gun Rights Policy Conference will stream online.

https://media.townhall.com/townhall/reu/o/2018/238/93bb559c-1119-49a5-b73c-c0aafefe3063-1110×740.jpg

September 29, 2022

A Factory That Makes Rice Cookers from Stone

https://theawesomer.com/photos/2022/09/making_rice_cookers_from_stone_t.jpg

Link

Modern rice cookers are made from metal and plastic, but traditional Korean rice cookers are made from stone. This fascinating video from Factory Monster takes us inside a company that creates the bowl-shaped cookers and their lids by cutting them from a 9-ton boulder. The English subtitles are quite entertaining.

The Awesomer

September 29, 2022

Are You Nuts? Know your Fishing Knots! – The Non-Slip Loop Knot

https://www.alloutdoor.com/wp-content/uploads/2022/09/20220920_171915.jpg

Last week we covered the Surgeon Knot, one of the quickest and easiest loop knots to tie. But there is a downside to that knot, it’s a bit bulky and especially in a more finesse presentation isn’t the best choice for anglers. This is especially true for fly anglers chasing after skittish trout. This is where the Non-Slip Loop Knot comes into the picture. This knot does take more effort to tie but will be a much neater knot once you’re finished with it. The Non-Slip Loop Knot has comparable strength to the Surgeon knot and works great for direct attachment to lures, jigs, and flies.

Step 1

Get your mainline and tie an overhand knot in it, making sure to have a decent length of tag end left for tying the rest of the knot.