11+ Laravel Tips: Optimize Database Queries (2024)

https://websolutionstuff.com/adminTheme/assets/img/11_laravel_tips_optimize_database_queries_2024.jpg

Hey developers! If you’re like me, constantly striving to make your Laravel applications faster and more efficient, you’re in for a treat. In this guide, I’m excited to share 11+ game-changing Laravel tips to supercharge your database queries as we step into 2024.

Database optimization doesn’t have to be a head-scratcher, and with these simple tips, we’ll explore ways to enhance your Laravel projects, ensuring they not only run smoothly but also deliver top-notch performance.

In this article, we’ll see 11+ laravel tips: optimize database queries (2024), the best 11 tips and tricks to improve database queries in laravel 8/9/10, and query optimization in laravel 2024.

Ready to dive into the world of optimized database queries? Let’s make our Laravel applications faster and more responsive together.

1. Minimizing Unnecessary Queries

Sometimes, we end up running database queries that aren’t really needed. Take a look at the example below.

<?php
 
class PostController extends Controller
{
    public function index()
    {
        $posts = Post::all();
        $private_posts = PrivatePost::all();
        return view('posts.index', ['posts' => $posts, 'private_posts' => $private_posts ]);
    }
}

The provided code fetches rows from two distinct tables (e.g., "posts" and "private_posts") and then sends them to a view. Take a peek at the corresponding view file presented below.

// posts/index.blade.php
 
@if( request()->user()->isAdmin() )
    <h2>Private Posts</h2>
    <ul>
        @foreach($private_posts as $post)
            <li>
                <h3></h3>
                <p>Published At: </p>
            </li>
        @endforeach
    </ul>
@endif
 
<h2>Posts</h2>
<ul>
    @foreach($posts as $post)
        <li>
            <h3></h3>
            <p>Published At: </p>
        </li>
    @endforeach
</ul>

As you can see above, $private_posts is visible to only a user who is an admin. Rest all the users cannot see these posts.

We can modify our logic below to avoid this extra query.

$posts = Post::all();
$private_posts = collect();
if( request()->user()->isAdmin() ){
    $private_posts = PrivatePost::all();
}

 

2. Consolidate Similar Queries for Improved Efficiency

Sometimes, we find ourselves needing to create queries to fetch various types of rows from a single table.

$published_posts = Post::where('status','=','published')->get();
$featured_posts = Post::where('status','=','featured')->get();
$scheduled_posts = Post::where('status','=','scheduled')->get();

Instead of this 3 different queries:

$posts =  Post::whereIn('status',['published', 'featured', 'scheduled'])->get();
$published_posts = $posts->where('status','=','published');
$featured_posts = $posts->where('status','=','featured');
$scheduled_posts = $posts->where('status','=','scheduled');

 

 

3. Optimizing Performance: Adding Index to Frequently Queried Columns

When you’re filtering queries using a condition on a text-based column, it’s a smart move to slap an index on that column. Why? Because adding an index makes your queries way speedier when sifting through rows.

Think of it like a well-organized filing system – it just makes finding what you need a whole lot faster!

$posts = Post::where('status','=','published')->get();

In the example above, we’re fetching records based on a condition added to the "status" column. To boost the query’s performance, consider enhancing it with the following database migration.

Schema::table('posts', function (Blueprint $table) {
   $table->index('status');
});

 

4. Optimize Pagination: Switch to simplePaginate Over Paginate

When it comes to paginating results, our typical approach would be:

$posts = Post::paginate(10);

When using pagination in Laravel, the typical approach involves two queries: one to retrieve paginated results and another to count the total number of rows in the table. Counting rows can be slow and impact query performance.

But why does Laravel count the total number of rows?

It does so to generate pagination links. By knowing the total number of pages beforehand, along with the current page number, Laravel facilitates easy navigation. You can jump to any page with confidence.

On the flip side, using simplePaginate skips the total row count, making the query faster. However, you sacrifice the knowledge of the last page number and the ability to jump to specific pages.

For large database tables, favor simplePaginate over paginate can significantly improve performance.

$posts = Post::paginate(20); // Generates pagination links for all the pages

$posts = Post::simplePaginate(20); // Generates only next and previous pagination links

 

5. Optimizing Database Queries: Avoiding Leading Wildcards with the LIKE Keyword

When aiming to retrieve results that match a particular pattern, our usual go-to approach is to use:

select * from table_name where column like %keyword%

The previous query scans the entire table, which can be inefficient. If we’re aware that the keyword appears at the start of the column value, a more efficient query can be formulated as follows:

select * from table_name where column like keyword%

 

6. Optimizing WHERE Clauses: Minimizing the Use of SQL Functions

It’s advisable to steer clear of using SQL functions in the WHERE clause, as they can lead to a full table scan. Take a peek at the example below: when querying results based on a specific date, the typical approach involves:

$posts = POST::whereDate('created_at', '>=', now() )->get();

This will result in a query similar to below.

select * from posts where date(created_at) >= 'timestamp-here'

The initial query causes a full table scan because the where condition isn’t applied until the date function is evaluated.

To improve this, we can restructure the query to eliminate the need for the date SQL function, as shown below:

$posts = Post::where('created_at', '>=', now() )->get();
select * from posts where created_at >= 'timestamp-here'

 

7. Optimizing Table Structure: Minimizing the Addition of Excessive Columns

To enhance performance, it’s wise to keep the number of columns in a table to a minimum. In databases like MySQL, you can optimize by breaking down tables with numerous columns into multiple tables. These tables can then be linked using primary and foreign keys.

Including excessive columns in a table extends the length of each record, leading to slower table scans. This becomes evident when executing a "select *" query, as it fetches unnecessary columns, causing a slowdown in retrieval speed

 

8. Separating Columns with Text Data Type into Their Own Table

When dealing with tables that store substantial data, especially in columns like TEXT, it’s wise to consider separating them into their own table or into a less frequently accessed table.

This practice proves beneficial because columns with extensive information can significantly inflate the size of individual records, impacting query times.

For instance, picture a table named "posts" with a "content" column storing hefty blog post content. Given that this detailed content is typically required only when someone is viewing that specific blog post, extracting this column from the main "posts" table can dramatically enhance query performance, especially when dealing with a multitude of posts.

 

9. More Efficient Method for Retrieving the Latest Rows from a Table

When we aim to fetch the most recent rows from a table, our usual approach often involves the following:

$posts = Post::latest()->get();
// or $posts = Post::orderBy('created_at', 'desc')->get();

The above approach will produce the following SQL query.

select * from posts order by created_at desc

Instead of this, you can do like this:

$posts = Post::latest('id')->get();
// or $posts = Post::orderBy('id', 'desc')->get();
select * from posts order by id desc

 

10. Optimizing MySQL Inserts

So far, we’ve focused on making select queries faster for fetching data from a database. Usually, our attention revolves around optimizing read queries. Yet, there are instances where we need to speed up insert and update queries as well.

// Instead of inserting records one by one like this:
foreach ($data as $record) {
    DB::table('your_table')->insert($record);
}

// You can optimize it by using the insert method with an array of data like this:
DB::table('your_table')->insert($data);

 

11. Inspecting and Optimizing Queries

When it comes to optimizing queries in Laravel, there’s no one-size-fits-all solution. After all, who knows your application better than you do? Understanding its behavior, the number of queries it churns out, and which ones are necessary is key.

By inspecting these queries, you gain valuable insights and can work towards reducing their overall number.

To aid in this crucial task, several tools are available to help you scrutinize queries on every page.

However, a word of caution: refrain from running these tools in your production environment. Doing so might compromise your application’s performance and expose sensitive information to unauthorized users.

Here are a few tools to inspect and optimize your queries:

  1. Laravel Debugbar:

    • Laravel Debugbar features a handy "database" tab, revealing all executed queries when you navigate through your pages. Visit each page in your application to observe the queries in action.
  2. Clockwork:

    • Similar to Laravel Debugbar, Clockwork provides debug information. However, instead of injecting a toolbar into your website, it displays the details in the developer tools window or as a standalone UI accessible at yourappurl/clockwork.
  3. Laravel Telescope:

    • Laravel Telescope serves as an excellent debugging companion during local Laravel development. Once installed, access the dashboard by visiting yourappurl/telescope. Navigate to the "queries" tab to view and analyze all the queries executed by your application.

Remember, these tools are best suited for your development environment to fine-tune your queries without risking your production’s performance and security.

Happy optimizing!

 


You might also like:

Laravel News Links

Deadpool 3’s Trailer Is Here to Save the Marvel Cinematic Universe

https://i.kinja-img.com/image/upload/c_fill,h_675,pg_1,q_80,w_1200/e88bbe770aef551ed635c3449e39cb66.jpg

The first trailer for Deadpool 3 is here.
Image: Marvel Studios

Marvel Studios is only releasing one movie this year—but, from the looks of it, it’s going to be unforgettable. That movie, of course, is Deadpool & Wolverine, which brings the R-rated, fourth-wall-breaking hero from Fox’s X-Men Universe into the Marvel Cinematic Universe. Ryan Reynolds stars and, this time, he’s bringing along his friend Hugh Jackman as Wolverine.

io9 Interview: Oscar Isaac Was ‘All In’ on Moon Knight

Directed by Shawn Levy, Deadpool 3 is one of the most highly anticipated Marvel films in years and now the first trailer is here.

Filming only wrapped a few weeks ago, so the fact that we’re getting a trailer at all for this is pretty incredible. So, what do you think?

Deadpool 3 opens in theaters July 26.

[Editor’s Note: This article is part of the developing story. The information cited on this page may change as the breaking story unfolds. Our writers and editors will be updating this article continuously as new information is released. Please check this page again in a few minutes to see the latest updates to the story. Alternatively, consider bookmarking this page or sign up for our newsletter to get the most up-to-date information regarding this topic.]

Read more from io9:


Want more io9 news? Check out when to expect the latest Marvel, Star Wars, and Star Trek releases, what’s next for the DC Universe on film and TV, and everything you need to know about the future of Doctor Who.

Gizmodo

USS Texas: “The Most Gangsta Battleship Of All Time”

http://img.youtube.com/vi/3oJSRAFkJIs/0.jpg

The Fat Electrician has a tribute video for USS Texas (which is still undergoing refurbishment).

  • “Today we’re talking about the most gangsta battleship of all time: The USS Texas, predating both World Wars, being built in 1914.”
  • “It’s commonly referred to as the last dreadnought. But it’s not technically a dreadnought, belonging to the New York-class of battleswhips, which were commonly referred to as super-dreadnoughts.” It was a class of two, with only the New York and the Texas. There was a pre-dreadnought USS Texas laid down in 1889 and scrapped in 1911.
  • “They had the largest guns ever put on a boat up to that time. That would be the Mark 1, capable of launching two 14 inch shells that weighed nearly 1,600 pounds apiece. The USS Texas had five of them, two in the front and three in the back. It was like a freedom sedan.”
  • They also had ballistic calculators and analogue computers, making them the most accurate naval guns in the world at the time. Plus a whole bunch of smaller guns.
  • “It was the first ship in history to incorporate anti-aircraft guns.”
  • “As well as having a 12″ thick hull, an entire freedom foot of Pittsburgh steel. The only thing millimeters is going to do to that is scratch the paint.”
  • It was the first ship to have a compliment of Marines onboard. “They let the water grunts drive the biggest gun ever made.”
  • The USS Texas saw “almost no combat” in World War I. Via Wikipedia: “Texas’s service with the Grand Fleet consisted entirely of convoy missions and occasional forays to reinforce the British squadron on blockade duty in the North Sea whenever German heavy units threatened.”
  • “But it’s actions in World War II made it a naval legend.” Lots of newer, more powerful ships than the Texas, but the Texas was the only battleship to engage the enemy in all five theaters.
  • “D-Day, June 6, 1944. The Texas would take it’s position 12,000 yards off the coast of Normandy.” It fired 235 rounds at German fortifications in just under 54 minutes. “That is four hundred and eight thousand pounds of ammunition.”
  • “The Texan was shooting the enemy with about three spicy Volvos a minute.”
  • “I’m trying to tell you the Grim Yeeter over here bitchslapped the enemy’s coastline with an entire car dealership in the amount of time it takes you to watch a TV show.”
  • Continued bombarding until running out of ammo June 11, at which point it went back for resupply. By the time it was back, allied troops had driven the enemy so far inland its guns couldn’t reach. So it moved in to 3,000 yards, the closest it could get without beaching the ship.
  • “It’s at that point the Texas said ‘Hold my beer’ and flooded all the blister tanks on the starboard side, tilting the entire boat, changing the angle of the guns, allowing them to reach further inland.”
  • “They gangster leaned a 32 thousand ton warship so they could continue to engage the enemy. This might be the most grunterrific moment in world history.”
  • “It’s not technically a war crime. Geneva didn’t even necessarily know that shit was a fucking option.”
  • The Texas would go on to fight at Okinawa and Iwo Jima.
  • More info at https://battleshiptexas.org/.

    Lawrence Person’s BattleSwarm Blog

    True dat

    https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6P4fZeQ-ErUMLngDE8R37Vevq7jqsTEzgAymMbDY8fLneS66tKW18bZZqNm0kx_mHXxkt1FbIfrOU2faZVVTJxe3st1mkU-2eOpMVQj8OQU1pyZGXa0E0XSXjTzzFc66iarqx7PSQb2GuBFmspDFEyFrS7S2LEatAYNAAxG8qMvz0CVMaOk6_lfE16Fw/w400-h356/Thug%20culture%20problem.png

     

    Found on social media:

    True dat.

    Peter

    Bayou Renaissance Man

    Laravel – Eager loading can be bad!

    Laravel – Eager loading can be bad!

    January 28, 2024

    Hello ????




    cover el

    Yes, you read it right. Eager loading can be bad, really bad. However, we often resort to it when dealing with an N+1 scenario, thinking that we’ve resolved the issue, when in fact, we might have made it worse. How? Let’s see.

    How bad it gets

    For this demo, we are building Laravel Forge. Like (almost) every Laravel application, we will have a One To Many relationship.

    We aim to log every activity for a server. A log can include the activity type, the user who initiated it, and other useful information for later analysis.

    <?php
    namespace App\Models;
    use Illuminate\Database\Eloquent\Model;
    use Illuminate\Database\Eloquent\Relations\HasMany;
    class Server extends Model
    {
    public function logs(): HasMany
    {
    return $this->hasMany(Log::class);
    }
    }

    Now, in the application, we want to list all the servers. So, we might do something like

    
    
    <table>
        <tr>
            <th>Name</th>
        </tr>
        @foreach ($servers as $server)
        <tr>
            <td></td>
        </tr>
        @endforeach
    </table>

    Moving forward, we have 10 servers, and each of them has 1000 logs.

    So far, so good. Now, we want to display when the last activity on a server occurred

    <table>
        <tr>
            <th>Name</th>
            <th>Last Activity</th>
        </tr>
        @foreach ($servers as $server)
        <tr>
            <td></td>
            <td>
                
            </td>
        </tr>
        @endforeach
    </table>

    Basic things, we access the logs() relation, ordering it to retrieve the latest record, getting the created_at column, and formatting it for better readability using diffForHumans(). The latter yields something like "1 week ago".

    But this is bad, we’ve introduced an N+1 problem.

    If you don’t know what a N+1 is, we are running the following queries

    
    select * from `servers`
    
    
    select * from `logs` where `logs`.`server_id` = 1 and `logs`.`server_id` is not null order by `created_at` desc limit 1
    select * from `logs` where `logs`.`server_id` = 2 and `logs`.`server_id` is not null order by `created_at` desc limit 1
    
    select * from `logs` where `logs`.`server_id` = 10 and `logs`.`server_id` is not null order by `created_at` desc limit 1

    To resolve this issue, we typically reach out to Eager Loading (I know you did).

    
    $servers = Server::query()
        ->with('logs')
        ->get();
    
    
    <table>
        <tr>
            <th>Name</th>
            <th>Last Activity</th>
        </tr>
        @foreach ($servers as $server)
        <tr>
            <td>{{ $server->name }}</td>
            <td>
                {{ $server->logs->sortByDesc('created_at')->first()->created_at->diffForHumans() }}
            </td>
        </tr>
        @endforeach
    </table>

    With this update, we manage to reduce it to only 2 queries

    
    select * from `servers`
    
    
    select * from `logs` where `logs`.`server_id` in (1, 2, 3, 4, 5, 6, 7, 8, 9, 10)

    And it looks like we addressed the problem, right?

    Wrong! We’re only considering the number of queries. Let’s examine the memory usage and the count of loaded models; these factors are equally important.

    • Before eager loading
      • 11 queries: 1 to retrieve all servers and 10 queries for each server.
      • A total of 20 models loaded.
      • Memory usage: 2MB.
      • Execution time: 38.19 ms.




    before

    • After eager loading
      • 2 queries: 1 to get all servers and 1 to get all logs.
      • A total of 10010 models loaded ????.
      • Memory usage: 13MB (6.5x increase).
      • Execution time: 66.5 ms (1.7x increase).
      • Slower computational time due to loading all the models ????.




    after

    The tool in the screenshot is Debugbar.

    Looks like we didn’t fix anything; in fact, we made it worse.. And keep in mind, this is a very simplified example. In a real world scenario, you can easily end up with hundreds or thousands of records, leading to the loading of millions of models.. The title makes sense now?

    How do we truly solve this?

    In our case, eager loading is a NO NO. Instead, we can use sub-queries and leverage the database to perform tasks it is built and optimized for.

    $servers = Server::query()
        ->addSelect([
            'last_activity' => Log::select('created_at')
                ->whereColumn('server_id', 'servers.id')
                ->latest()
                ->take(1)
        ])
        ->get();

    This will result in a single query

    select `servers`.*, (
            select `created_at`
            from `logs`
            where
                `server_id` = `servers`.`id`
            order by `created_at` desc
            limit 1
        ) as `last_activity`
    from `servers`

    Since the column we need from the relationship is now computed in a subquery, we have the best of both worlds: only 10 models loaded and minimal memory usage.

    You might be thinking that with this approach comes a drawback: the last_activity column is now a regular string. So, if you want to use the diffForHumans() method, you’ll encounter the Call to a member function diffForHumans() on string error. But no worries, you haven’t lost the casting; it’s as simple as adding a single line.

    $servers = Server::query()
        ->addSelect([
            'last_activity' => Log::select('created_at')
                ->whereColumn('server_id', 'servers.id')
                ->latest()
                ->take(1)
        ])
        ->withCasts(['last_activity' => 'datetime']) 
        ->get();

    By chaining the withCasts() method, you can now treat the last_activity as if it were a date.

    How about the Laravel way?

    The reddit community never disappoints! They have pointed out another alternative solution, a Laravel-ish approach; One Of Many.

    Let’s define a new relationship to always retrieve the latest log

    
    public function latestLog(): HasOne
    {
        return $this->hasOne(Log::class)->latestOfMany();
    }
    

    Now we can use the relationship like this

    
    $servers = Server::query()
        ->with('latestLog')
        ->get();
    

    This will result in the following queries

    select * from `servers`
    
    select `logs`.*
    from
        `logs`
        inner join (
            select MAX(`logs`.`id`) as `id_aggregate`, `logs`.`server_id`
            from `logs`
            where
                `logs`.`server_id` in (1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
            group by
                `logs`.`server_id`
        ) as `latestOfMany` 
        on `latestOfMany`.`id_aggregate` = `logs`.`id`
        and `latestOfMany`.`server_id` = `logs`.`server_id`

    And it can be used in the Blade like this

    
    @foreach ($servers as $server)
        {{$server->latestLog }}
    @endforeach

    For a comparison between the two methods:

    • Using subqueries
      • 1 query.
      • A total of 10 models loaded.
      • Memory usage: 2MB.
      • Execution time: 21.55 ms.




    old

    • Using the latestOfMany()
      • 2 queries
      • A total of 20 models loaded.
      • Memory usage: 2MB.
      • Execution time: 20.63 ms




    new

    Both methods are really good; which one to use will depend on your case. If you absolutely need the child model hydrated and will make use of all its fields, go with the latestOfMany(). However, if you only need a few fields, then the subquery will perform better. This is because, in the subquery, you select exactly what you need. Regardless of the number of records you have, the memory usage will be almost the same. Now, for the second method, memory usage is heavily dependent on the number of columns your table has. In reality, a table can easily have 50 columns, so hydrating the model will be expensive, even if it is only one per parent, that is to keep in mind when choosing!

    Conclusion

    I have seen some developers, by design, choose to force eager loading for all the models. You can’t just use it for everything, as much as it seems like you’ve solved the issue, you might have actually created a worse one. Not everything is a nail; the hammer might not work ????


    Laravel News Links

    Watch Timothée Chalamet Ride a Sandworm in Extended Dune: Part Two Clip

    https://i.kinja-img.com/image/upload/c_fill,h_675,pg_1,q_80,w_1200/eb0ccae04f5ca6091e1591c68777727d.jpg

    Dune: Part Two is finally just a few weeks away from hitting theaters, and a new sneak peek lifts the lid on one of the sequel’s most eagerly awaited scenes: Paul Atreides (Timothée Chalamet) riding a sandworm on Arrakis.

    Spoilers of the Week April 24-29

    This clip shared by Fandango fully reveals the first time Paul rides a sandworm—you easily get a sense of how thrilling (in the moment) and important (to Paul’s journey as he becomes a great leader) the scene is. It’s very cool and exciting to watch. However, you have to assume you won’t really get the full sensory experience until you’re watching Denis Villeneuve’s film on the biggest screen possible… preferably with one of those viral Dune: Part Two popcorn buckets clutched in your hands.

    Still, sandworm-riding still looks rather amazing, even when it’s scaled down to fit your phone or computer screen, don’t you think?

    Dune: Part Two hits theaters March 1.


    Want more io9 news? Check out when to expect the latest Marvel, Star Wars, and Star Trek releases, what’s next for the DC Universe on film and TV, and everything you need to know about the future of Doctor Who.

    Gizmodo

    Build E-Commerce System in Seconds With TomatoPHP

    https://media.dev.to/cdn-cgi/image/width=1000,height=500,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F19c0mf70jkiwwgaqv7x9.png

    hi, community.

    Introducing #tomatophp, a cutting-edge open-source Laravel package designed to streamline development within the VILT stack environment. This innovative ecosystem leverages the power of Splade to effortlessly generate modern, high-performance single-page applications (SPAs) using only Blade files. #tomatophp redefines the development experience, making it both efficient and enjoyable.

    As we progress, we continuously enhance our ecosystem by incorporating additional features into our plugins. Presently, we have developed a comprehensive e-commerce system using these plugins. In this article, we will guide you through the implementation process, demonstrating how you can seamlessly integrate a robust e-commerce system into your application.



    Install Tomato Admin

    you need to install tomato-admin plugin on your fresh Laravel app so let’s create a new Laravel app.

    composer create-project laravel/laravel tomato
    

    if you don’t have an environment for Laravel you can use this doc to build one on your Ubuntu Linux.

    now cd inside your project folder change .env of your database and make sure that your Laravel app is running and the database is connected, you can check that by running migrations

    php artisan migrate
    

    now let’s start installing tomato-admin

    composer require tomatophp/tomato-admin
    

    after the composer is done run this command for auto-install

    php artisan tomato-admin:install
    

    if you are using macOS you can easily use auto yarn package install if not just build your assets like this

    yarn & yarn build
    

    now you have tomato-admin installed on your Laravel project.

    we will use some media on our package to we need to publish Spatie Media Library migrations

    php artisan vendor:publish --provider="Spatie\MediaLibrary\MediaLibraryServiceProvider" --tag="medialibrary-migrations"
    

    if this command does not work please use this command

    Image description

    php artisan vendor:publish
    

    then on the search type media and select the migration one.

    now migrate your files

    php artisan migrate
    

    you can check your browser now and you will see a homepage like this

    Image description

    we need to change HOME const on the RouteServiceProvider.php to be /admin to make the redirect after auth to admin.



    Install Tomato Roles

    No e-commerce system is complete without a robust role management structure. To fulfill this essential requirement, we’ll be installing ‘tomato-roles’ to seamlessly handle roles within our mission to build a comprehensive e-commerce system.

    composer require tomatophp/tomato-roles
    

    after the composer is done run this command

    php artisan tomato-roles:install
    

    now go to your app\Models\User.php and add this trait to it

    use \Spatie\Permission\Traits\HasRoles;
    

    now your dashboard is ready to log in using admin@admin.com and password as a password from this URL /admin/login

    Image description

    if you try to access any page you will be redirected to Two-factor Confirmation If you don’t go it for now, you can easily stop it by removing implements MustVerifyEmail from your User.php model.



    Install Tomato CRM

    As an integral part of our e-commerce system, the management of customer interactions, authentications, and other crucial actions is paramount. To efficiently handle these aspects, we’ll be installing ‘tomato-crm.’ Let’s proceed with the installation to empower our system with advanced customer relationship management capabilities.

    composer require tomatophp/tomato-crm
    

    now let’s install it

    php artisan tomato-crm:install
    

    let’s publish Accounts.php model to our app to custom it

    php artisan vendor:publish --tag="tomato-crm-model"
    

    and you need to publish tomato-crm config

    php artisan vendor:publish --tag="tomato-crm-config"
    

    on tomato-crm.php config change the model path to like this

    "model" => \App\Models\Account::class,
    

    now we need to add a new guard to our app, so let’s add it on the config auth.php like this.

    <?php
    
    return [
    
        /*
    |--------------------------------------------------------------------------
    | Authentication Defaults
    |--------------------------------------------------------------------------
    |
    | This option controls the default authentication "guard" and password
    | reset options for your application. You may change these defaults
    | as required, but they're a perfect start for most applications.
    |
    */
    
        'defaults' => [
            'guard' => 'web',
            'passwords' => 'users',
        ],
    
        /*
    |--------------------------------------------------------------------------
    | Authentication Guards
    |--------------------------------------------------------------------------
    |
    | Next, you may define every authentication guard for your application.
    | Of course, a great default configuration has been defined for you
    | here which uses session storage and the Eloquent user provider.
    |
    | All authentication drivers have a user provider. This defines how the
    | users are actually retrieved out of your database or other storage
    | mechanisms used by this application to persist your user's data.
    |
    | Supported: "session"
    |
    */
    
        'guards' => [
            'web' => [
                'driver' => 'session',
                'provider' => 'users',
            ],
            'accounts' => [
                'driver' => 'session',
                'provider' => 'accounts',
            ]
        ],
    
        /*
    |--------------------------------------------------------------------------
    | User Providers
    |--------------------------------------------------------------------------
    |
    | All authentication drivers have a user provider. This defines how the
    | users are actually retrieved out of your database or other storage
    | mechanisms used by this application to persist your user's data.
    |
    | If you have multiple user tables or models you may configure multiple
    | sources which represent each model / table. These sources may then
    | be assigned to any extra authentication guards you have defined.
    |
    | Supported: "database", "eloquent"
    |
    */
    
        'providers' => [
            'users' => [
                'driver' => 'eloquent',
                'model' => App\Models\User::class,
            ],
            'accounts' => [
                'driver' => 'eloquent',
                'model' => App\Models\Account::class,
            ],
        ],
    
        /*
    |--------------------------------------------------------------------------
    | Resetting Passwords
    |--------------------------------------------------------------------------
    |
    | You may specify multiple password reset configurations if you have more
    | than one user table or model in the application and you want to have
    | separate password reset settings based on the specific user types.
    |
    | The expiry time is the number of minutes that each reset token will be
    | considered valid. This security feature keeps tokens short-lived so
    | they have less time to be guessed. You may change this as needed.
    |
    | The throttle setting is the number of seconds a user must wait before
    | generating more password reset tokens. This prevents the user from
    | quickly generating a very large amount of password reset tokens.
    |
    */
    
        'passwords' => [
            'users' => [
                'provider' => 'users',
                'table' => 'password_reset_tokens',
                'expire' => 60,
                'throttle' => 60,
            ],
        ],
    
        /*
    |--------------------------------------------------------------------------
    | Password Confirmation Timeout
    |--------------------------------------------------------------------------
    |
    | Here you may define the amount of seconds before a password confirmation
    | times out and the user is prompted to re-enter their password via the
    | confirmation screen. By default, the timeout lasts for three hours.
    |
    */
    
        'password_timeout' => 10800,
    
    ];
    

    now clear your config cache

    php artisan config:clear
    

    now rebuild your assets

    yarn & yarn build
    

    now your CRM is ready you can check it on your dashboard



    Install Tomato Wallet

    For seamless transaction management between customers and vendors within the e-commerce system, a robust payment handler is crucial. Introducing ‘tomato-wallet,’ a feature-rich package that not only manages customer wallets but also seamlessly handles payments. Packed with a multitude of integrated payment gateways, ‘tomato-wallet’ ensures a magical experience in managing transactions. Let’s proceed with the installation to unlock the full potential of this powerful payment solution.

    composer require tomatophp/tomato-wallet
    

    let’s install it

    php artisan tomato-wallet:install
    

    now we need to implement Wallet interface to our Account.php and add HasWallet trait to it to make the wallet of the customer work, your Account model must be like this

    <?php
    
    namespace App\Models;
    
    use Bavix\Wallet\Interfaces\Wallet;
    use Bavix\Wallet\Traits\HasWallet;
    use Illuminate\Database\Eloquent\Factories\HasFactory;
    use Illuminate\Database\Eloquent\Model;
    use Illuminate\Foundation\Auth\User as Authenticatable;
    use Illuminate\Notifications\Notifiable;
    use Laravel\Sanctum\HasApiTokens;
    use Spatie\Macroable\Macroable;
    use Spatie\MediaLibrary\HasMedia;
    use Spatie\MediaLibrary\InteractsWithMedia;
    use Spatie\Permission\Traits\HasRoles;
    use TomatoPHP\TomatoCrm\Models\Group;
    
    /**
    * @property integer $id
    * @property string $name
    * @property string $username
    * @property string $loginBy
    * @property string $address
    * @property string $type
    * @property string $password
    * @property string $otp_code
    * @property string $otp_activated_at
    * @property string $last_login
    * @property string $agent
    * @property string $host
    * @property integer $attempts
    * @property boolean $login
    * @property boolean $activated
    * @property boolean $blocked
    * @property string $deleted_at
    * @property string $created_at
    * @property string $updated_at
    * @property AccountsMeta[] $accountsMetas
    * @property Activity[] $activities
    * @property Comment[] $comments
    * @property Model meta($key, $value)
    * @property Location[] $locations
    */
    class Account extends Authenticatable implements HasMedia, Wallet
    {
        use InteractsWithMedia;
        use HasApiTokens, HasFactory, Notifiable;
        use HasWallet;
    
        /**
    * @var array
    */
        protected $fillable = [
            'email',
            'phone',
            'parent_id',
            'type',
            'name',
            'username',
            'loginBy',
            'address',
            'password',
            'otp_code',
            'otp_activated_at',
            'last_login',
            'agent',
            'host',
            'is_login',
            'is_active',
            'deleted_at',
            'created_at',
            'updated_at'
        ];
    
        protected $casts = [
            'is_login' => 'boolean',
            'is_active' => 'boolean'
        ];
        protected $dates = [
            'deleted_at',
            'created_at',
            'updated_at',
            'otp_activated_at',
            'last_login',
        ];
    
    
        protected $appends = [
            'birthday',
            'gender',
            'more'
        ];
    
        public function getMoreAttribute()
        {
            $metas = $this->accountsMetas()->get()->pluck('value', 'key')->toArray();
            return $metas;
        }
    
        public function getBirthdayAttribute()
        {
            return $this->meta('birthday') ?: null;
        }
    
        public function getGenderAttribute()
        {
            return $this->meta('gender') ?: null;
        }
    
        /**
    * @return \Illuminate\Database\Eloquent\Relations\HasMany
    */
        public function accountsMetas()
        {
            return $this->hasMany('TomatoPHP\TomatoCrm\Models\AccountsMeta');
        }
    
        /**
    * @param string $key
    * @param string|null $value
    * @return Model|string
    */
        public function meta(string $key, string|null $value=null): Model|string|null
        {
            if($value){
                return $this->accountsMetas()->updateOrCreate(['key' => $key], ['value' => $value]);
            }
            else {
                return $this->accountsMetas()->where('key', $key)->first()?->value;
            }
        }
        /**
    * @return \Illuminate\Database\Eloquent\Relations\HasMany
    */
        public function activities()
        {
            return $this->hasMany('TomatoPHP\TomatoCrm\Models\Activity');
        }
    
        /**
    * @return \Illuminate\Database\Eloquent\Relations\HasMany
    */
        public function comments()
        {
            return $this->hasMany('TomatoPHP\TomatoCrm\Models\Comment');
        }
    
        /**
    * @return \Illuminate\Database\Eloquent\Relations\HasMany
    */
        public function locations()
        {
            return $this->hasMany('TomatoPHP\TomatoCrm\Models\Location');
        }
    
        public function groups(){
            return $this->belongsToMany(Group::class, 'account_groups', 'account_id', 'group_id');
        }
    }
    

    now your wallet is working you can start any transaction and check the customer’s balance.



    Install Tomato CMS

    In any functioning e-commerce setup, a well-optimized front end is indispensable. To enhance SEO performance and add essential content such as posts and pages, we’ll be incorporating our ‘tomato-cms’ package. Let’s proceed with the installation to seamlessly integrate this package and elevate our e-commerce platform.

    composer require tomatophp/tomato-cms
    

    now let’s install it

    php artisan tomato-cms:install
    



    Install Tomato Themes

    To craft a dynamic frontend tailored as a theme, we’ll be utilizing the ‘tomato-themes’ package. This package simplifies the implementation of multi-themes for your project, employing a Hierarchical Model-View-Controller (HMVC) architecture. Let’s initiate the installation process for ‘tomato-themes’ to facilitate the seamless integration of diverse themes into your project.

    composer require tomatophp/tomato-themes
    

    now let’s install it

    php artisan tomato-themes:install
    

    now rebuild your assets

    yarn & yarn build
    

    make sure that you have Themes folder in your project root and add this line to your composer.json

     "autoload": {
    "psr-4": {
    "App\\": "app/",
    "Database\\Factories\\": "database/factories/",
    "Database\\Seeders\\": "database/seeders/",
    "Themes\\": "Themes/"
    }
    },
    

    now you need to reload the composer

    composer dump-autoload
    

    now inside your tailwind.config.js add these lines

    content: [
        ...
        "./Themes/**/*.blade.php",
        "./Themes/**/**/*.blade.php",
    ],
    

    now your theme is ready to upload or create a new theme.



    Install Tomato E-Commerce

    Now that we have our foundational elements in place, including a ready CRM, role management, and several essential packages, it’s time to bring it all together by installing the E-Commerce System. The seamless integration of these components promises a robust and feature-rich platform. Let’s proceed with the installation to witness the culmination of our efforts in building a comprehensive and efficient E-Commerce System.

    composer require tomatophp/tomato-ecommerce
    

    let’s install it

    php artisan tomato-ecommerce:install
    

    now everything i ready to install our e-commerce theme.

    this package will install tomato-products, tomato-orders, tomato-offers, tomato-branches for you

    to make everything work fine you need some actions, we need to install tomato-branches

    php artisan tomato-branches:install
    

    after that, you need to create just 1 Shipping Vendor from this endpoint /admin/shipping-vendors

    now you need to change your site SEO data and your site Logo from this endpoint /admin/settings/seo upload your logos and change the Site Name.



    Install E-commerce Theme

    Having established a solid foundation, including CRM, role management, and various essential packages, we’re now ready to enhance the visual appeal of our E-Commerce and CMS platforms. Introducing our user-friendly theme with a simple yet stylish design – an ideal canvas for customization. Let’s embark on the installation process to seamlessly integrate this theme and provide users the flexibility to tailor it to their preferences.

    inside your Themes Folder clone this repo

    cd Themes
    

    now clone our Theme.

    git clone git@github.com:tomatophp/Ecommerce.git
    

    now go to your dashboard /themes and you will get the new theme you can just activate it.

    Please note that you must not have / route on your main routes/web.php because these routes can override the Theme Routes.

    to fix style rebuild your assets

    yarn & yarn build
    

    now if you check your home page you will get something like this

    Image description

    you can select from the top dropdown any section and add it to your page.

    and you can build a menu from /admin/menus using this endpoint

    • Home /
    • About /about
    • Shop /shop
    • Blog /blog
    • Contact /contact
    • Terms & Conditions /terms
    • Privacy /privacy

    you can create 2 menus main, footer the main will show up auto to your header and the footer to your footer.

    change the middleware of Authenticate redirect to this route

    <?php
    
    namespace App\Http\Middleware;
    
    use Illuminate\Auth\Middleware\Authenticate as Middleware;
    use Illuminate\Http\Request;
    
    class Authenticate extends Middleware
    {
        /**
    * Get the path the user should be redirected to when they are not authenticated.
    */
        protected function redirectTo(Request $request): ?string
        {
            return $request->expectsJson() ? null : route('accounts.login');
        }
    }
    

    now in your Account.php model add this trait

        use \TomatoPHP\TomatoEcommerce\Services\Traits\InteractsWithEcommerce;
        use \TomatoPHP\TomatoNotifications\Traits\InteractWithNotifications;
        use \TomatoPHP\TomatoOrders\Services\Traits\InteractsWithOrders;
    

    With the successful installation of our comprehensive E-commerce system, you are now equipped to seamlessly manage your product catalog. Begin by adding new products and defining categories to tailor your offerings. Feel empowered to kickstart the order creation process, as your E-commerce system stands ready to facilitate smooth transactions and streamline your online business operations.

    ???? Thanks for using Tomato Plugins & TomatoPHP framework
    ???? Join the support server on Discord here
    ???? You can check docs here
    ⭐ Please give us a star on any repo if you like it TomatoPHP GitHub
    ???? Sponsor us here

    Laravel News Links

    Web Development Is Getting Too Complex, And It May Be Our Fault

    http://files.smashing.media/articles/web-development-getting-too-complex/web-development-getting-too-complex.jpg

    Front-end development seemed simpler in the early 2000s, didn’t it? The standard website consisted mostly of static pages made of HTML and CSS seasoned with a pinch of JavaScript and jQuery. I mean, who doesn’t miss the cross-browser compatibility days, right?

    Fast forward to today, and it looks like a parallel universe is taking place with an overwhelming number of choices. Which framework should you use for a new project? Perhaps more established ones like React, Angular, Vue, Svelte, or maybe the hot new one that came out last month? Each framework comes with its unique ecosystem. You also need to decide whether to use TypeScript over vanilla JavaScript and choose how to approach server-side rendering (or static site generation) with meta-frameworks like Next, Nuxt, or Gatsby. And we can’t forget about unit and end-to-end testing if you want a bug-free web app. And we’ve barely scratched the surface of the front-end ecosystem!

    But has it really gotten more complex to build websites? A lot of the frameworks and tooling we reach for today were originally crafted for massive projects. As a newcomer, it can be frightening to have so many to consider, almost creating a fear of missing out that we see exploited to sell courses and tutorials on the new hot framework that you “cannot work without.”

    All this gives the impression that web development has gotten perhaps too complex. But maybe that is just an exaggeration? In this article, I want to explore those claims and find out if web development really is that complex and, most importantly, how we can prevent it from getting even more difficult than we already perceive it to be.

    How It Was Before

    As someone who got into web development after 2010, I can’t testify to my own experience about how web development was from the late 1990s through the 2000s. However, even fifteen years ago, learning front-end development was infinitely simpler, at least to me. You could get a website started with static HTML pages, minimal CSS for styling, and a sprinkle of JavaScript (and perhaps a touch of jQuery) to add interactive features, from toggled sidebars to image carousels and other patterns. Not much else was expected from your average developer beyond that — everything else was considered “going the extra mile.” Of course, the awesome native CSS and JavaScript features we have today weren’t around back then, but they were also unnecessary for what was considered best practice in past years.

    Large and dynamic web apps certainly existed back then — YouTube and Facebook, to name a couple — but they were developed by massive companies. No one was expected to re-create that sort of project on their own or even a small team. That would’ve been the exception rather than the norm.

    I remember back then, tend to worry more about things like SEO and page optimization than how my IDE was configured, but only to the point of adding meta tags and keywords because best practices didn’t include minifying all your assets, three shaking your code, caching your site on edge CDNs, or rendering your content on the server (a problem created by modern frameworks along hydration). Other factors like accessibility, user experience, and responsive layouts were also largely overlooked in comparison to today’s standards. Now, they are deeply analyzed and used to boost Lighthouse scores and impress search engine algorithms.

    The web and everything around it changed as more capabilities were added and more and more people grew to depend on it. We have created new solutions, new tools, new workflows, new features, and whatever else new that is needed to cater to a bigger web with even bigger needs.

    The web has always had its problems in the past that were worthy of fixing: I absolutely don’t miss tables and float layouts, along with messy DOM manipulation. This post isn’t meant to throw shade on new advances while waxing nostalgic about the good days of the “old wild web.” At the same time, though, yesterday’s problems seem infinitely simpler than those we face today.

    JavaScript Frameworks

    JavaScript frameworks, like Angular and React, were created by Google and Facebook, respectively, to be used in their own projects and satisfy the needs that only huge web-based companies like them have. Therein lies the main problem with web complexity: JavaScript frameworks were originally created to sustain giant projects rather than smaller ones. Many developers vastly underestimate the amount of time it takes to build a codebase that is reliable and maintainable with a JavaScript framework. However, the alternative of using vanilla JavaScript was worse, and jQuery was short for the task. Vanilla JavaScript was also unable to evolve quickly enough to match our development needs, which changed from simple informative websites to dynamic apps. So, many of us have quickly adopted frameworks to avoid directly mingling with JavaScript and its messy DOM manipulation.

    Back-end development is a completely different topic, subject to its own complexities. I only want to focus on front-end development because that is the discipline that has perhaps overstepped its boundaries the most by bleeding into traditional back-end concerns.

    Stacks Getting Bigger

    It was only logical for JavaScript frameworks to grow in size over time. The web is a big place, and no one framework can cover everything. But they try, and the complexity, in turn, increases. A framework’s size seems to have a one-to-one correlation with its complexity.

    But the core framework is just one piece of a web app. Several other technologies make up what’s known as a tech “stack,” and with the web gaining more users and frameworks catering to their needs, tech stacks are getting bigger and bigger. You may have seen popular stacks such as MEAN (MongoDB, Express, Angular, and Node) or its React (MERN) and Vue (MEVN) variants. These stacks are marketed as mature, test-proofed foundations suitable for any front-end project. That means the advertised size of a core framework is grossly underestimated because they rely on other micro-frameworks to ensure highly reliable architectures, as you can see in stackshare.io. Besides, there isn’t a one-size-fits-all stack; the best tool has always depended — and will continue to depend — on the needs and goals of your particular project.

    This means that each new project likely requires a unique architecture to fulfill its requirements. Giant tech companies need colossal architectures across all their projects, and their stacks are highly engineered accordingly to secure scalability and maintenance. They also have massive customer bases, so maintaining a large codebase will be easier with more revenue, more engineers, and a clearer picture of the problem. To minimize waste, the tech stacks of smaller companies and projects can and should be minimized not only to match the scale of their needs but to the abilities of the developers on the team as well.

    The idea that web development is getting too complex comes from buying into the belief that we all have the same needs and resources as giant enterprises.

    Trying to imitate their mega stacks is pointless. Some might argue that it’s a sacrifice we have to make for future scalability and maintenance, but we should focus first on building great sites for the user without worrying about features users might need in the future. If what we are building is worth pursuing, it will reach the point where we need those giant architectures in good time. Cross that bridge when we get there. Otherwise, it’s not unlike wearing Shaquille O’Neal-sized sneakers in hopes of growing into them. They might not even last until then if it happens at all!

    We must remember that the end-user experience is the focus at the end of the day, and users neither care about nor know what stack we use in our apps. What they care about is a good-looking, useful website where they can accomplish what they came for, not the technology we use to achieve it. This is how I’ve come to believe that web development is not getting more complex. It’s developers like us who are perpetuating it by buying into solutions for problems that do not need to be solved at a certain scale.

    Let me be really clear: I am not saying that today’s web development is all bad. Indeed, we’ve realized a lot of great features, and many of them are thanks to JavaScript frameworks that have pushed for certain features. jQuery had that same influence on JavaScript for many, many years.

    We can still create minimum viable products today with minimal resources. No, those might not make people smash the Like button on your social posts, but they meet the requirements, nothing more and nothing less. We want bigger! Faster! Cheaper! But we can’t have all three.

    If anything, front-end development has gotten way easier thanks to modern features that solve age-old development issues, like the way CSS Flexbox and Grid have trivialized layouts that used to require complex hacks involving floats and tables. It’s the same deal with JavaScript gaining new ways to build interactions that used to take clever workarounds or obtuse code, such as having the Intersection Observer API to trivialize things like lazy loading (although HTML has gained its own features in that area, too).

    We live in this tension between the ease of new platform features and the complexity of our stacks.

    Do We Need A JavaScript Framework For Everything?

    Each project, regardless of its simplicity, desperately needs a JavaScript framework. A project without a complex framework is like serving caviar on a paper plate.

    At least, that’s what everyone seems to think. But is that actually true? I’d argue on the contrary. JavaScript frameworks are best used on bigger applications. If you’re working on a smaller project, a component-based framework will only complicate matters, making you split your website into a component hierarchy that amounts to overkill for small projects.

    The idea of needing a framework for everything has been massively oversold. Maybe not directly, but you unconsciously get that feeling whenever a framework’s name pops in, as Edge engineer Alex Russell eloquently expresses in his article, “The Market For Lemons”:

    “These technologies were initially pitched on the back of “better user experiences” but have utterly failed to deliver on that promise outside of the high-management-maturity organisations in which they were born. Transplanted into the wider web, these new stacks have proven to be expensive duds.”

    — Alex Russell

    Remember, the purpose of a framework is to simplify your life and save time. If the project you’re working on is smaller, the time you supposedly save is likely overshadowed by the time you spend either setting up the framework or making it work with the rest of the project. A framework can help make bigger web apps more interactive and dynamic, but there are times when a framework is a heavy-handed solution that actually breeds inefficient workflows and introduces technical debt.

    Step back and think about this: Are HTML, CSS, and a touch of JavaScript enough to build your website or web application? If so, then stick with those. What I am afraid of is adding complexity for complexity’s sake and inadvertently raising the barrier to entry for those coming into web development. We can still accomplish so much with HTML and CSS alone, thanks again to many advances in the last decade. But we give the impression that they are unsuitable for today’s web consumption and need to be enhanced.

    Knowing Everything And Nothing At The Same Time

    The perceived standard that teams must adopt framework-centered architectures puts a burden not only on the project itself but on a developer’s well-being, too. As mentioned earlier, most teams are unable to afford those architectures and only have a few developers to maintain them. If we undermine what can be achieved with HTML and CSS alone and set the expectations that any project — regardless of size — needs to have a bleeding edge stack, then the weight to meet those expectations falls on the developer’s shoulders, with the great responsibility of being proficient in all areas, from the server and database to front end, to design, to accessibility, to performance, to testing, and it doesn’t stop. It’s what has been driving “The Great Divide” in front-end development, which Chris Coyier explains like this:

    “The divide is between people who self-identify as a (or have the job title of) front-end developer yet have divergent skill sets. On one side, an army of developers whose interests, responsibilities, and skillsets are heavily revolved around JavaScript. On the other, an army of developers whose interests, responsibilities, and skillsets are focused on other areas of the front end, like HTML, CSS, design, interaction, patterns, accessibility, and so on.”

    — Chris Coyier

    Under these expectations, developers who focus more on HTML, CSS, design, and accessibility rather than the latest technology will feel less valued in an industry that appears to praise those who are concerned with the stack. What exactly are we saying when we start dividing responsibilities in terms of “full-stack development” or absurd terms like “10x development”? A while back, Brad Frost began distinguishing these divisions as “front-of-the-front-end” and “back-of-the-front-end”.

    Mandy Michael explains what impact the chase for “full-stack” has had on developers trying to keep up:

    “The worst part about pushing the “know everything” mentality is that we end up creating an industry full of professionals suffering from burnout and mental illness. We have people speaking at conferences about well-being, imposter syndrome, and full-stack anxiety, yet despite that, we perpetuate this idea that people have to know everything and be amazing at it.”

    — Mandy Michael

    This isn’t the only symptom of adopting heavy-handed solutions for what “vanilla” HTML, CSS, and JavaScript already handle nicely. As the expectations for what we can do as front-end developers grow, the learning curve of front-end development grows as well. Again, we can’t learn and know everything in this vast discipline. But we tell ourselves we have to, and thanks to this mentality, it’s unfortunately common to witness developers who may be extremely proficient with a particular framework but actually know and understand little of the web platform itself, like HTML semantics and structure.

    The fact that many budding developers tend to jump straight into frameworks at the expense of understanding the basics of HTML and CSS isn’t a new worry, as Rachel Andrew discussed back in 2019:

    “That’s the real entry point here, and yes, in 2019, they are going to have to move on quickly to the tools and techniques that will make them employable, if that is their aim. However, those tools output HTML and CSS in the end. It is the bedrock of everything that we do, which makes the devaluing of those with real deep skills in those areas so much more baffling.”

    — Rachel Andrew

    And I want to clarify yet again that modern Javascript frameworks and libraries aren’t inherently bad; they just aren’t designed to replace the web platform and its standards. But we keep pushing them like we want them to!

    The Consequences Of Vendor Lock-In

    “Vendor lock-in” happens when we depend too deeply on proprietary products and services to the extent that switching to other products and services becomes a nearly impossible task. This often occurs when cloud services from a particular company are deeply integrated into a project. It’s an issue, especially in cloud computing, since moving databases once they are set up is expensive and lengthy.

    Vendor lock-in in web development has traditionally been restricted to the back end, like with cloud services such as AWS or Firebase; the front-end framework, meanwhile, was a completely separate concern. That said, I have noticed a recent trend where vendor lock-in is reaching into meta-frameworks, too. With the companies behind certain meta-frameworks offering hosting services for their own products, swapping hosts is increasingly harder to do (whether the lock-in is designed intentionally or not). Of course, companies and developers will be more likely to choose the hosting service of the company that made a particular framework used on their projects — they’re the experts! — but that only increases the project’s dependency on those vendors and their services.

    A clear example is the relationship between Next and Vercel, the parent cloud service for Next. With the launch of Next 13, it has become increasingly harder to set up a Next project outside of Vercel, leading to projects like Open Next, which says right on its website that “[w]hile Vercel is great, it’s not a good option if all your infrastructure is on AWS. Hosting it in your AWS account makes it easy to integrate with your backend [sic]. And it’s a lot cheaper than Vercel.” Fortunately, the developers’ concerns have been heard, and Next 14 brings clarity on how to self-host Next on a Node server.

    Another example is Gatsby and Gatsby Cloud. Gatsby has always offered helpful guides and alternative hosting recommendations, but since the launch of Gatsby Cloud in 2019, the main framework has been optimized so that using Gatsby and Gatsby Cloud together requires no additional hosting configurations. That’s fantastic if you adopt both, but it’s not so great if all you need is one or the other because integrating the framework with other hosts — and vice versa — is simply harder. It’s as if you are penalized for exercising choice.

    And let’s not forget that no team expected Netlify to acquire Gatsby Cloud in February 2023. This is a prime case where the vendor lock-in problem hits everybody because converting from one site to another comes at a cost. Some teams were charged 120% more after converting from Gatsby Cloud to Netlify — even with the same plan they had with Gatsby Cloud!

    What’s the solution? The common answer I hear is to stop using paid cloud services in favor of open-sourced alternatives. While that’s great and indeed a viable option for some projects, it fails to consider that an open-source project may not meet the requirements needed for a given app.

    And even then, open-source software depends on the community of developers that maintain and update the codebase with little to no remuneration in exchange. Further, open source is equally prone to locking you into certain solutions that are designed to solve a deficiency with the software.

    There are frameworks and libraries, of course, that are in no danger of being abandoned. React is a great example because it has an actively engaged community behind it. But you can’t have the same assurance with each new dependency you add to a project. We can’t simply keep installing more packages and components each time we spot a weak spot in the dependency chain, especially when a project is perfectly suited for a less complex architecture that properly leverages the web platform.

    Choosing technology for your stack is an exercise of picking your own poison. Either choose a paid service and be subject to vendor lock-in in the future, or choose an open-source one and pray that the community continues to maintain it.

    Those are virtually the only two choices. Many of the teams I know or have worked on depend on third-party services because they cannot afford to develop them on their own; that’s a luxury that only massive companies can afford. It’s a problem we have to undergo when starting a new project, but one we can minimize by reducing the number of dependencies and choosing wisely when we have to.

    Each Solution Introduces A New Problem

    Why exactly have modern development stacks gotten so large and complex? We can point a finger at the “Development Paradox.” With each new framework or library, a new problem crops up, and time-starved developers spend months developing a new tool to solve that problem. And when there isn’t a problem, don’t worry — we will create one eventually. This is a feedback loop that creates amazing solutions and technologies but can lead to over-engineered websites if we don’t reign it in.

    This reminds me of the famous quote:

    “The plain fact is that if you don’t have a problem, you create one. If you don’t have a problem, you don’t feel that you are living.”

    — U.G. Krishnamurti

    Let’s look specifically at React. It was originally created by Facebook for Facebook to develop more dynamic features for users while improving Facebook’s developer experience.

    Since React was open-sourced in 2013 (and nearly re-licensed in 2017, if it weren’t for the WordPress community), hundreds of new utilities have been created to address various React-specific problems. How do you start a React project? There’s Create React App and Vite. Do you need to enhance your state management? There is Redux, among other options. Need help creating forms? There is a React Hook Form. And perhaps the most important question: Do you need server-side rendering? There’s Next, Remix, or Gatsby for that. Each solution comes with its own caveats, and developers will create their own solutions for them.

    It may be unfair to pick on React since it considers itself a library, not a framework. It’s inevitably prone to be extended by the community. Meanwhile, Angular and Vue are frameworks with their own community ecosystems. And this is the tip of the iceberg since there are many JavaScript frameworks in the wild, each with its own distinct ideology and dependencies.

    Again, I don’t want you to get the wrong idea. I love that new technologies emerge and find it liberating to have so many options. But when building something as straightforward as a webpage or small website — which some have started referring to as “multi-page applications” — we have to draw a line that defines how many new technologies we use and how reliable they are. We’re quite literally mashing together third-party code written by various third-party developers. What could go wrong? Please don’t answer that.

    Remember that our users don’t care what’s in our stacks. They only see the final product, so we can save ourselves from working on unnecessary architectures that aren’t appreciated outside of development circles. It may seem counterintuitive in the face of advancing technology, but knowing that the user doesn’t care about what goes behind the scenes and only sees the final product will significantly enhance our developer experience and free you from locked dependencies. Why fix something that isn’t broken?

    How Can We Simplify Our Codebases?

    We’ve covered several reasons why web development appears to be more complex today than in years past, but blaming developers for releasing new utilities isn’t an accurate portrayal of the real problem. After all, when developing a site, it’s not like we are forced to use each new technology that enters the market. In fact, many of us are often unaware of a particular library and only learn about it when developing a new feature. For example, if we want to add toast notifications to our web app, we will look for a library like react-toastify rather than some other way of building them because it “goes with” that specific library. It’s worth asking whether the app needs toast notifications at all if they introduce new dependencies.

    Imagine you are developing an app that allows users to discover, review, and rate restaurants in their area. The app needs, at a bare minimum, information about each restaurant, a search tool to query them, and an account registration flow with authentication to securely access the account. It’s easy to make assumptions about what a future user might need in addition to these critical features. In many cases, a project ends up delayed because we add unnecessary features like SSR, notifications, offline mode, and fancy animations — sometimes before the app has even converted its first registered user!

    I believe we can boil down the complexity problem to personal wishes and perceived needs rather than properly scoping a project based on user needs and experiences.

    That level of scope creep can easily turn into an over-engineered product that will likely never see the light of launching.

    What can we do to simplify our own projects? The following advice is relevant when you have control over your project, either because it’s a personal one, it’s a smaller one for a smaller team, or you have control over the decisions in whatever size organization you happen to be in.

    The hardest and most important step is having a sense of detection when your codebase is getting unnecessarily complicated. I deem it the hardest step because there is no certainty of what the requirements are or what the user needs; we can only make assumptions. Some are obvious, like assuming the user will need a way to log into the app. Others might be unclear, like whether the app should have private messaging between users. Others are still far-fetched, like believing users need extremely low latency in an e-commerce page. Other features are in the “nice to have” territory.

    That is regarding the user experience, but the same questions emerge on the development side:

    • Should we be using a CSS preprocessor or a CSS framework, or can we achieve it using only CSS modules?
    • Is vanilla JavaScript enough, or are we going to add TypeScript?
    • Does the app need SSR, SSG, or a hybrid of the two?
    • Should we implement Redis on the back end for faster database queries, or is that too much scope for the work?
    • Should we be implementing end-to-end testing or unit tests?

    These are valid questions that should be considered when developing a site, but they can distract us from our main focus: getting things done.

    “Done is better than perfect.”

    — Sheryl Sandberg

    And, hey, even the largest and most sophisticated apps began as minimal offerings that iterated along the way.

    We also ought to be asking ourselves what would happen if a particular feature or dependency isn’t added to the project. If the answer is “nothing,” then we should be shifting our attention to something else.

    Another question worth asking: “Why are we choosing to add [X]?” Is it because that’s what is popular at the moment, or because it solves a problem affecting a core feature? Another aspect to take into consideration is how familiar we are with certain technologies and give preference to those we know and can start using them right away rather than having to stop and learn the ins and outs of a new framework.

    Choose the right tool for the job, which is going to be the one that meets the requirements and fits your mental model. Focus less on a library’s popularity and scalability but rather on getting your app to the point where it needs to scale in the first place.

    Conclusion

    It’s incredibly difficult to not over-engineer web apps given current one-size-fits-all and fear-of-missing-out mentalities. But we can be more conscious of our project goals and exercise vigilance in guarding our work against scope creep. The same can be applied to the stack we use, making choices based on what is really needed rather than focusing purely on what everyone else is using for their particular work.

    After reading the word “framework” exactly 48 times in this article, can we now say the web is getting too complex? It has been complex by nature since its origins, but complexity doesn’t translate to “over-engineered” web apps. The web isn’t intrinsically over-engineered, and we only have ourselves to blame for over-engineering our projects with overly-wrought solutions for perceived needs.

    Smashing Magazine