MySQL Workbench 8.0.14 has been released

Dear MySQL users, 

The MySQL developer tools team announces 8.0.14 as our general available (GA) for MySQL Workbench 8.0.

For the full list of changes in this revision, visit http://dev.mysql.com/doc/relnotes/workbench/en/changes-8-0.html

For discussion, join the MySQL Workbench Forums: http://forums.mysql.com/index.php?152

The release is now available in source and binary form for a number of platforms from our download pages at: http://dev.mysql.com/downloads/tools/workbench/

Enjoy!

via Planet MySQL
MySQL Workbench 8.0.14 has been released

The Latest Shazam TV Spot is Chock Full of New Footage


Shazam is definitely, really, absolutely an adult. Legally. Yes.
Image: Warner Bros.

This kid is about to become a super man.

In the latest TV spot for Shazam, young Billy Batson is doing the things any kid would do if he was granted super powers that transformed him into a magically gifted adult man. He’s flying! He’s transforming! He’s trying to buy a super lair! And he’s buying beer. Or trying to, anyways. I doubt Shazam’s costume comes with an ID.

It looks like an incredibly fun time, an offbeat sort of childish humor in tension both with grimdark superheroism and the snarkfest of the MCU. I’m sold. This is charming. Kid heroes are the best.


For more, make sure you’re following us on our new Instagram @io9dotcom.


via Gizmodo
The Latest Shazam TV Spot is Chock Full of New Footage

Five Hidden Features of the Laravel Excel Package



News
/
updated: January 18, 2019

Five Hidden Features of the Laravel Excel Package

The Laravel Excel package recently celebrated a new milestone of version 3, with new features that help ease advanced use-cases, and is simple to use. Let’s explore some of these hidden features you might not know about, that make Laravel Excel a go-to package for working with Excel.

1. Exporting from HTML/Blade

Let’s imagine you already have a list page with HTML table:

And here’s the Blade code – resources/views/customers/table.blade.php:

<table class="table">
    <thead>
    <tr>
        <th></th>
        <th>First name</th>
        <th>Last name</th>
        <th>Email</th>
        <th>Created at</th>
        <th>Updated at</th>
    </tr>
    </thead>
    <tbody>
    @foreach ($customers as $customer)
    <tr>
        <td></td>
        <td></td>
        <td></td>
        <td></td>
        <td></td>
        <td></td>
    </tr>
    @endforeach
    </tbody>
</table>

You can re-use it to export the same table into Excel.

Step 1. Generate Export class

php artisan make:export CustomersFromView --model=Customer

Step 2. Use FromView to perform the operation.

namespace App\Exports;

use App\Customer;
use Illuminate\Contracts\View\View;
use Maatwebsite\Excel\Concerns\FromView;

class CustomersExportView implements FromView
{
    public function view(): View
    {
        return view('customers.table', [
            'customers' => Customer::orderBy('id', 'desc')->take(100)->get()
        ]);
    }
}

Here’s the result Excel file:

Notice: you can export only HTML table, without any layout tags like html, body, div, etc.


2. Export to PDF, HTML, and others

Although the package is called Laravel Excel, it provides export to more formats. It’s straightforward to use, add one more parameter to the class:

return Excel::download(new CustomersExport(), 'customers.xlsx', 'Html');

Yes, you got it right. HTML. Here’s how it looks:

Not much styling, I know. And here’s the source:

Not only that, it allows to export to PDF, and you can even choose from three libraries for it. Again, all you need to do is specify the format as the last parameter – here’s screenshot from their docs:

Notice: you will also have to install a chosen PDF package via composer, like:

composer require dompdf/dompdf

Here’s how PDF looks like:


3. Format Cells However You Want

Laravel Excel package has a powerful “parent” – PhpSpreadsheet. So it adopts all the underneath functionality, including cell formatting in various ways.

Here’s how to use it in Laravel Export class, like app/Exports/CustomersExportStyling.php:

Step 1. Use appropriate classes in the header.

use Maatwebsite\Excel\Concerns\WithEvents;
use Maatwebsite\Excel\Events\AfterSheet;

Step 2. Use WithEvents in implements section.

class CustomersExportStyling implements FromCollection, WithEvents 
{ 
    // ...

Step 3. Create registerEvents() method with AfterSheet event.

/**
 * @return array
 */
public function registerEvents(): array
{
    return [
        AfterSheet::class    => function(AfterSheet $event) {
            // ... HERE YOU CAN DO ANY FORMATTING
        },
    ];
}

Here’s an example:

/**
 * @return array
 */
public function registerEvents(): array
{
    return [
        AfterSheet::class    => function(AfterSheet $event) {
            // All headers - set font size to 14
            $cellRange = 'A1:W1'; 
            $event->sheet->getDelegate()->getStyle($cellRange)->getFont()->setSize(14);

            // Apply array of styles to B2:G8 cell range
            $styleArray = [
                'borders' => [
                    'outline' => [
                        'borderStyle' => \PhpOffice\PhpSpreadsheet\Style\Border::BORDER_THICK,
                        'color' => ['argb' => 'FFFF0000'],
                    ]
                ]
            ];
            $event->sheet->getDelegate()->getStyle('B2:G8')->applyFromArray($styleArray);

            // Set first row to height 20
            $event->sheet->getDelegate()->getRowDimension(1)->setRowHeight(20);

            // Set A1:D4 range to wrap text in cells
            $event->sheet->getDelegate()->getStyle('A1:D4')
                ->getAlignment()->setWrapText(true);
        },
    ];
}

The result of these “random” demo-styling examples looks like this:

All these examples, and many more, you can find in Recipes page of PhpSpreadsheet docs.


4. Hidden Fields From Model

Let’s imagine we’ve seeded a default Laravel 5.7 users table:

Let’s try to export this one with a simple class FromCollection:

class UsersExport implements FromCollection
{
    public function collection()
    {
        return User::all();
    }
}

In the result Excel, you will see some fields missing: password and remember_token:

This is because of this app/User.php property:

class User extends Authenticatable
{
    // ...

    /**
     * The attributes that should be hidden for arrays.
     *
     * @var array
     */
    protected $hidden = [
        'password', 'remember_token',
    ];
}

So, these fields are hidden by default, but it shows us Laravel Excel package behavior – if you want to protect some fields from export, you can do it directly in the model.


5. Formulas

For some reason, official Laravel Excel package documentation doesn’t mention anything about formulas. But that’s the whole point of using Excel!

Luckily, it’s straightforward to write formulas into our exported file. We need to set cell values like we would to in Excel, for example, =A2+1 or SUM(A1:A10).

One of the ways to do it is to use WithMapping:

use App\Customer;
use Maatwebsite\Excel\Concerns\FromCollection;
use Maatwebsite\Excel\Concerns\WithMapping;

class CustomersExportFormulas implements FromCollection, WithMapping
{
    public function collection()
    {
        return Customer::all();
    }

    /**
     * @var Customer $customer
     * @return array
     */
    public function map($customer): array
    {
        return [
            $customer->id,
            '=A2+1',
            $customer->first_name,
            $customer->last_name,
            $customer->email,
        ];
    }
}

So, these are just five less-known features of Laravel Excel. If you want to find out more, I have a unique online course called Excel Export/Import in Laravel, go check it out!


via Laravel News
Five Hidden Features of the Laravel Excel Package

NSA Puts Phone Charging Station at Hacker Conference in Plot to Go Viral


We’ve all had that moment. You smartphone battery is running low, and you’re desperate for some juice. But just how desperate have you gotten? Desperate enough to plug your phone into an NSA charging station? Because some people are seriously facing that choice right now.

Shmoocon, a hacker conference that started today and runs through the weekend, includes a cutesy display with this exact dilemma. The text on the NSA’s display even says, “You know you want to try it!”

NSA official Rob Joyce tweeted out a photo of the display (above), and the spy agency says that it’s in on the joke.

“It’s tongue-in-cheek,” NSA media relations officer Chris Augustine told Gizmodo over the phone. “It was created for this event so that it would allow for… exactly what you’re seeing on Twitter.”

And what you’re seeing on Twitter is a tweet that’s starting to go viral with comments like, “what could go wrong?” and “totally not a trap.”

“We’re trying be edgy,” Augustine continues with a laugh.

But if you’re heading to a hacker conference, you’re probably going to want to lock your shit down. And that includes not charging your devices with mystery cords. Even if NSA says it’s just a joke.

“We go to a lot of conferences like these,” Augustine continued. “Some of our talent comes from conferences like these. I don’t if we’re presenting at this conference, but I do know we are recruiting.”


via Gizmodo
NSA Puts Phone Charging Station at Hacker Conference in Plot to Go Viral

Here’s the action-packed first trailer for John Wick: Chapter 3—Parabellum


Keanu Reeves is on the run with his trusty canine companion in first trailer for John Wick: Chapter 3—Parabellum.

Everyone’s favorite reluctant assassin is on the run with a $14 million bounty on his head, and few allies, in the action-packed first trailer for John Wick: Chapter 3—Parabellum.

(Spoilers for first two movies below.)

For those who missed the first two movies in the trilogy, John Wick (Keanu Reeves) is a legendary hitman (known as “Baba Yaga”) who tried to retire when he fell in love and got married. Unfortunately, he’s drawn back into the dark underground world by an act of senseless violence after his wife’s death. As Wick mourns Helen’s passing, Iosef Tarasov, the son of a Russian crime syndicate, breaks in, kicks him unconscious, and steals his classic 1969 Ford Mustang Mach 1. On top of all that, Tarasov kills the little dog, Daisy, that Helen gave to John to comfort him. From there, there’s really no hope for Iosef. Nothing will stop John Wick from seeking retribution.

The first John Wick doubled its original projected box office on opening weekend and went on to grow more than $88 million worldwide for a film that cost around $30 million to make. It received praise for its brisk pace, heart-stopping action sequences, and stylish noir feel. Reeves was perfectly cast in the lead role, and the fictional underground culture of assassins with their gold coins, markers, and almost medieval code of honor set the film apart from more typical entries in this genre.

So naturally, there was a sequel—which was still pretty good but not quite as riveting as its predecessor. John Wick: Chapter Two was set about four days after the events of the first film ended, with Wick exacting revenge and picking up a new pit bull puppy for good measure. In this edition, he wants his Mustang back, and who can blame him? He finds it at a Russian syndicate chop shop and ends up taking out a whole slew of Russian baddies (and badly damaging the car) before declaring a truce.

  • Where it all began: John’s late wife, Helen, is why he tried to retire.


    YouTube/Lionsgate

  • He’s still got his trusty pit bull with him, even on the run.


    YouTube/Lionsgate

  • Anjelica Huston is The Director, a member of the High Table.


    YouTube/Lionsgate

  • Even the administrative staff is on tenterhooks.


    YouTube/Lionsgate

  • “Here we go.” Winston (Ian McShane, owner of the Continental Hotel in New York, prepares to make John Wick “excommunicado.”


    YouTube/Lionsgate

  • Winston and Charon (Lance Reddick), concierge of the Continental Hotel, assess John’s odds. “I’d say they’re about even.”


    YouTube/Lionsgate

  • First assassin spotted in Grand Central Station.


    YouTube/Lionsgate

  • A motorcycle chase with sword-bearing assassins.


    YouTube/Lionsgate

  • Sorry, would-be assassin, John Wick on a horse is still better than you.


    YouTube/Lionsgate

  • Laurence Fishburne plays The Bowery King, an underground crime lord.


    YouTube/Lionsgate

  • Would-be assassins wisely on their guard.


    YouTube/Lionsgate

  • Shooting at a reflection isn’t the most effective defense.


    YouTube/Lionsgate

  • Battling it out in what amounts to a hall of mirrors.


    YouTube/Lionsgate

  • If you come at John Wick, you’d best not miss with that knife, bud.


    YouTube/Lionsgate

  • Halle Berry plays fellow assassin Sofia, a close friend of John Wick.


    YouTube/Lionsgate

  • Must Love Dogs. Sofia shares John’s affection for canine friends.


    YouTube/Lionsgate

  • Master assassin in action.


    YouTube/Lionsgate

  • Here’s hoping these two crazy kids can work things out.


    YouTube/Lionsgate

  • Fired up and ready to take on all comers.


    YouTube/Lionsgate

But there is no rest for a legendary hitman. An Italian crime lord, Santino D’Antonio (Riccardo Scamarcio), presents him with a “blood oath” marker Wick gave him in exchange for a favor before he retired. Under the assassin’s code of honor, Wick cannot refuse the request: to kill Santino’s sister, Gianna, so he can take her seat at the crime lord High Table. Santino then double-crosses him by opening a $7 million contract on his life, pretending it’s to avenge his sister (instead of trying to tie up loose ends).

Santino seeks sanctuary in the Continental Hotel, a safe space for assassins with a strict “no killing on the premises” policy. Wick kills Santino in the lounge anyway. And that puts hotel owner Winston (Ian McShane) on the spot. He has to enforce the policy, but he also likes John Wick and knows Santino had it coming. Winston thus gives Wick a one-hour head start before declaring him “excommunicado,” with no access to the hotel’s substantial underground resources. Since there’s also now a $14 million bounty on his head, the odds are heavily stacked against Wick’s survival.

This is where John Wick: Chapter 3—Parabellum picks up, with a countdown to Winston’s “excommunicado” declaration as Wick scrambles to find allies (and ammo). Based on the trailer, the third film looks likely to recapture the fast-paced, violent glory that made the original John Wick so irresistible. McShane is back as Winston, along with Lance Reddick as Charon, concierge of the Continental. You’ve got Anjelica Huston as The Director of the crime lord High Table—she’s a friend to Wick but unfortunately unable to offer much help since “the High Table wants your life.” Rounding out the cast, Halle Berry plays Sofia, another assassin and close friend of Wick’s, who might just be a potential love interest as well as a helpful ally.

If the film performs as well at the box office as its predecessors, who knows? We might just get a John Wick: Chapter 4. For now, fans of the first two will likely enjoy John Wick: Chapter 3—Parabellum when it hits theaters on May 17, 2019.

 

Listing image by YouTube/Lionsgate


via Ars Technica
Here’s the action-packed first trailer for John Wick: Chapter 3—Parabellum

The Best Duct Tape


Photo: Kyle Fitzgerald

Through our research, and backed up by our firsthand testing, we found that a good general use duct tape should be around 11 milli-inches (mil) thick, use a natural rubber-based adhesive, and be made using a co-extrusion process.

Before getting into these specifics, it helps to know a bit about the three ingredients that make up a piece of duct tape: a polyethylene sheet backing, a cloth grid, and a rubber-based adhesive.

The polyethylene sheet plays two roles: It serves as a bonding area for the other two ingredients and creates a waterproof backing. It’s basically a sheet of plastic.

The cloth grid (or scrim)—typically made of polyester or a cotton/polyester blend—decides the tape’s strength, flexibility, and tearability. The threads that run the length of the tape are what give it its material strength—how much weight the tape can hold before breaking. The threads that run across the tape determine its tearability. Duct tape tears along the thread line, so the smaller the space between threads, the cleaner the tear. If the threads are far apart, getting a straight, even tear is difficult. It’s like holding one edge of a piece of paper while trying to rip it down the center.

All true duct tape has a rubber-based adhesive, but each tape has its own adhesive recipe. Some are thicker and flow better in order to stick to rough porous surfaces, while others are stiffer, making them more stable for extreme temperatures and flat surfaces. As we found out in our testing, some adhesives are so gooey that they’ll melt in the hot sun. Some tapes are made with alternatives to rubber-based glues, but those tapes (often made with what’s known as hot melt adhesive) are less reliable in extreme temperatures, and they don’t have the strength of the rubber-based glues, so we dismissed those.

After many conversations with four prominent duct tape manufacturers (and confirmed through our testing), we’re convinced a process called coextrusion is the best way to assemble these three ingredients. The defining characteristic of coextrusion is that the polyethylene sheet enters the manufacturing process in molten form. This means that when the cloth grid is added, it melts directly into the plastic, forming a single, fully bonded piece. The rubber adhesive is then applied to one side, creating what we know as duct tape.

The other way to make duct tape is called lamination. It’s easier to do, but there are issues with the finished product. As Hillary DuMoulin, communications manager at Berry Plastics, explained to us, lamination involves pressing all three ingredients together. The cloth grid is held to the poly either by a separate laminating adhesive or the “squish-through” method, where the rubber-based adhesive holds everything together.

The problem with lamination is the poly/scrim connection is nowhere near as secure as it is with the coextrusion method. Air bubbles can form between the laminated layers. Over time, particularly during exterior use, the poly and scrim can come apart. If you’ve ever pulled off an old piece of duct tape and the cloth grid remained stuck in a crusty bed of adhesive, you’ve seen the major flaw of a laminated tape. With a co-extruded tape, the scrim is really an internal component of the poly, so this kind of separation doesn’t occur.

Two examples of delamination. The smaller pieces are an unknown tape that had been on the ends of two electrical conduits for the past three months. The larger piece is the tested Scotch All-Weather after two weeks on a plywood sample board. Photo: Doug Mahoney

There are visual ways to distinguish a laminated tape from a co-extruded one. The most telling is that co-extruded tapes have very small, clearly defined dimples on the exterior of the roll. These correspond with gaps in the cloth grid and represent all of the places where potential air bubbles could form if the tape were laminated. It’s harder to see on high-end tapes because as the tape quality increases, the grid gets smaller and dimples become difficult for the eye to pick up.

Another way to visually tell the difference is that laminated tapes (at least the two that we tested) have a wrinkled texture. On one tape, the ridges were so extreme that we were unable to get it to sit flat against any surface for more than a day or so. Co-extruded tapes have a nice smooth finish.

Notice the small dimples along the exterior of the co-extruded Duck Advanced (top). Also, the adhesive side of the laminated Scotch All-Weather (bottom) has an uneven adhesive coating and the scrim doesn’t appear to sit flat against the poly backing. Photo: Doug Mahoney

For thickness, we found that tapes around the 11-mil range offered the best compromise between strength and maneuverability. Thickness is measured in “mils” (or milli-inches, roughly .0254 millimeters) and tapes vary from as thin as 3 mils to as thick as 17 mils. Thinner tapes are too floppy to handle. Trying to wrap a thin, flimsy, wet noodle piece of duct tape can become a frustrating experience as it constantly folds over and sticks to itself. Once this happens, especially if it’s adhesive to adhesive, it’s pretty much a permanent bond, so you have to discard the piece and start over.

But thick tapes have their own drawbacks. The beefier they are, the less flexibility and conformity they have. During testing, we found that this negative outweighed the added strength of the really thick tapes, like the 17-mil Gorilla. Those tapes are extremely strong, but it’s very difficult to wrap a piece around a contoured surface (like the floppy sole on an old work boot, or the 90-degree elbow of a copper water pipe). This lack of flexibility also causes problems if you’re bundling something together with the tape. It’s much better if you can give the tape a strong pull and add a little stretch to it as you’re adhering it. This little bit of flexibility, which tapes around 11-mil usually have, can add just enough tension to secure the whole bundle together. This kind of stretching is nearly impossible to do with a thick 17-mil tape.

Other handling characteristics play a role in performance. Berry Plastics’ Hillary DuMoulin, told us that tearability, “finger tack,” and the ability to unwind are all important in evaluating duct tape. So during testing, we took these factors into account as well.

What’s not as important are width and length. The standard roll of duct tape is 2 inches wide (it actually measures around 1.88 inches but is always referred to as 2 inches). Other sizes are available, but even after 10 years in the construction industry, I’ve never needed anything more than a 2-inch roll. The lengths of the rolls are also standardized. The rolls I looked at were a variety of 35, 40, 45, and 60 yards. (45 and 60 yards are the most common lengths.)

Good, high-quality tapes designed for general purpose sit in the $8 to $13 range. You can get cheap and poorly made stuff for less, but it’s not worth saving the couple bucks unless it’s for a really simple and temporary job like sealing garbage bags or wrapping rolls of pulled-up carpeting. You can also get more aggressive tapes loaded with turbo strength and crazy adhesive that can cost over $20, but you probably don’t need that strength and you definitely don’t need the frustration that comes with working with tape that’s too thick and too sticky.

There is a dizzying variety of available duct tapes—Berry Plastics alone sells roughly 35 different types of duct tape under the Nashua and Polyken names—so after our research, we asked the major brands (Intertape, Duck, Scotch, Nashua, and Polyken) to suggest their best product for general all-around use. To give them a sense of what we had in mind, we gave the examples of patching a bike seat, repairing a backpack, and fixing a leak in a hose. Each company came back to us with a tape in the 9-to-11–mil range. In addition, we tested a number of tapes from these manufacturers based on reputation and customer feedback. These were typically thicker tapes and ones that tended to have specific strengths (and, as we discovered, specific weaknesses). We tested a total of 10 tapes.


via Wirecutter: Reviews for the Real World
The Best Duct Tape

Using Parallel Query with Amazon Aurora for MySQL

parallel query amazon aurora for mysqlParallel query execution is my favorite, non-existent, feature in MySQL. In all versions of MySQL – at least at the time of writing – when you run a single query it will run in one thread, effectively utilizing one CPU core only. Multiple queries run at the same time will be using different threads and will utilize more than one CPU core.

On multi-core machines – which is the majority of the hardware nowadays – and in the cloud, we have multiple cores available for use. With faster disks (i.e. SSD) we can’t utilize the full potential of IOPS with just one thread.

AWS Aurora (based on MySQL 5.6) now has a version which will support parallelism for SELECT queries (utilizing the read capacity of storage nodes underneath the Aurora cluster). In this article, we will look at how this can improve the reporting/analytical query performance in MySQL. I will compare AWS Aurora with MySQL (Percona Server) 5.6 running on an EC2 instance of the same class.

In Short

Aurora Parallel Query response time (for queries which can not use indexes) can be 5x-10x better compared to the non-parallel fully cached operations. This is a significant improvement for the slow queries.

Test data and versions

For my test, I need to choose:

  1. Aurora instance type and comparison
  2. Dataset
  3. Queries

Aurora instance type and comparison

According to Jeff Barr’s excellent article (https://aws.amazon.com/blogs/aws/new-parallel-query-for-amazon-aurora/) the following instance classes will support parallel query (PQ):

“The instance class determines the number of parallel queries that can be active at a given time:

  • db.r*.large – 1 concurrent parallel query session
  • db.r*.xlarge – 2 concurrent parallel query sessions
  • db.r*.2xlarge – 4 concurrent parallel query sessions
  • db.r*.4xlarge – 8 concurrent parallel query sessions
  • db.r*.8xlarge – 16 concurrent parallel query sessions
  • db.r4.16xlarge – 16 concurrent parallel query sessions”

As I want to maximize the concurrency of parallel query sessions, I have chosen db.r4.8xlarge. For the EC2 instance I will use the same class: r4.8xlarge.

Aurora:

mysql> show global variables like ‘%version%’;

+++

| Variable_name           | Value                        |

+++

| aurora_version          | 1.18.0                       |

| innodb_version          | 1.2.10                       |

| protocol_version        | 10                           |

| version                 | 5.6.10                       |

| version_comment         | MySQL Community Server (GPL) |

| version_compile_machine | x86_64                       |

| version_compile_os      | Linux                        |

+++

MySQL on ec2

mysql> show global variables like ‘%version%’;

+++

| Variable_name           | Value                                                |

+++

| innodb_version          | 5.6.4184.1                                          |

| protocol_version        | 10                                                   |

| slave_type_conversions  |                                                      |

| tls_version             | TLSv1.1,TLSv1.2                                      |

| version                 | 5.6.4184.1                                          |

| version_comment         | Percona Server (GPL), Release 84.1, Revision b308619 |

| version_compile_machine | x86_64                                               |

| version_compile_os      | debianlinuxgnu                                     |

| version_suffix          |                                                      |

+++

Table

I’m using the “Airlines On-Time Performance” database from http://www.transtats.bts.gov/DL_SelectFields.asp?Table_ID=236&DB_Short_Name=On-Time  (You can find the scripts I used here: https://github.com/Percona-Lab/ontime-airline-performance).

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

mysql> show table status like ‘ontime’\G

*************************** 1. row ***************************

          Name: ontime

        Engine: InnoDB

       Version: 10

    Row_format: Compact

          Rows: 173221661

Avg_row_length: 409

   Data_length: 70850183168

Max_data_length: 0

  Index_length: 0

     Data_free: 7340032

Auto_increment: NULL

   Create_time: 20180926 02:03:28

   Update_time: NULL

    Check_time: NULL

     Collation: latin1_swedish_ci

      Checksum: NULL

Create_options:

       Comment:

1 row in set (0.00 sec)

The table is very wide, 84 columns.

Working with Aurora PQ (Parallel Query)

Documentation: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-mysql-parallel-query.html

Aurora PQ works by doing a full table scan (parallel reads are done on the storage level). The InnoDB buffer pool is not used when Parallel Query is utilized.

For the purposes of the test I turned PQ on and off (normally AWS Aurora uses its own heuristics to determine if the PQ will be helpful or not):

Turn on and force:

mysql> set session aurora_pq = 1;

Query OK, 0 rows affected (0.00 sec)

mysql> set aurora_pq_force = 1;

Query OK, 0 rows affected (0.00 sec)

Turn off:

mysql> set session aurora_pq = 0;

Query OK, 0 rows affected (0.00 sec)

The EXPLAIN plan in MySQL will also show the details about parallel query execution statistics.

Queries

Here, I use the “reporting” queries, running only one query at a time. The queries are similar to those I’ve used in older blog posts comparing MySQL and Apache Spark performance (https://www.percona.com/blog/2016/08/17/apache-spark-makes-slow-mysql-queries-10x-faster/ )

Here is a summary of the queries:

  1. Simple queries:
    • select count(*) from ontime where flightdate > ‘2017-01-01’
    • select avg(DepDelay/ArrDelay+1) from ontime
  2. Complex filter, single table:

select SQL_CALC_FOUND_ROWS

FlightDate, UniqueCarrier as carrier, FlightNum, Origin, Dest

FROM ontime

WHERE

  DestState not in (‘AK’, ‘HI’, ‘PR’, ‘VI’)

  and OriginState not in (‘AK’, ‘HI’, ‘PR’, ‘VI’)

  and flightdate > ‘2015-01-01’

   and ArrDelay < 15

and cancelled = 0

and Diverted = 0

and DivAirportLandings = 0

  ORDER by DepDelay DESC

LIMIT 10;

3. Complex filter, join “reference” table

select SQL_CALC_FOUND_ROWS

FlightDate, UniqueCarrier, TailNum, FlightNum, Origin, OriginCityName, Dest, DestCityName, DepDelay, ArrDelay

FROM ontime_ind o

JOIN carriers c on o.carrier = c.carrier_code

WHERE

  (carrier_name like ‘United%’ or carrier_name like ‘Delta%’)

  and ArrDelay > 30

  ORDER by DepDelay DESC

LIMIT 10\G

4. select one row only, no index

Query 1a: simple, count(*)

Let’s take a look at the most simple query: count(*). This variant of the “ontime” table has no secondary indexes.

select count(*) from ontime where flightdate > ‘2017-01-01’;

Aurora, pq (parallel query) disabled:

I disabled the PQ first to compare:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

mysql> select count(*) from ontime where flightdate > ‘2017-01-01’;

++

| count(*) |

++

|  5660651 |

++

1 row in set (8 min 25.49 sec)

 

mysql> select count(*) from ontime where flightdate > ‘2017-01-01’;

++

| count(*) |

++

|  5660651 |

++

1 row in set (2 min 48.81 sec)

 

mysql> mysql> select count(*) from ontime where flightdate > ‘2017-01-01’;

++

| count(*) |

++

|  5660651 |

++

1 row in set (2 min 48.25 sec)

 

Please note: the first run was cold run; data was read from disk. The second and third run used the cached data.

 

Now lets enable and force Aurora PQ:

 

mysql> set session aurora_pq = 1;

Query OK, 0 rows affected (0.00 sec)

mysql> set aurora_pq_force = 1; 

Query OK, 0 rows affected (0.00 sec)

 

mysql> explain select count(*) from ontime where flightdate > ‘2017-01-01’\G

*************************** 1. row ***************************

          id: 1

 select_type: SIMPLE

       table: ontime

        type: ALL

possible_keys: NULL

         key: NULL

     key_len: NULL

         ref: NULL

        rows: 173706586

       Extra: Using where; Using parallel query (1 columns, 1 filters, 0 exprs; 0 extra)

1 row in set (0.00 sec)

(from the EXPLAIN plan, we can see that parallel query is used).

Results:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

mysql> select count(*) from ontime where flightdate > ‘2017-01-01’;                                                                                                                          

++

| count(*) |

++

|  5660651 |

++

1 row in set (16.53 sec)

 

mysql> select count(*) from ontime where flightdate > ‘2017-01-01’;

++

| count(*) |

++

|  5660651 |

++

1 row in set (16.56 sec)

 

mysql> select count(*) from ontime where flightdate > ‘2017-01-01’;

++

| count(*) |

++

|  5660651 |

++

1 row in set (16.36 sec)

 

mysql> select count(*) from ontime where flightdate > ‘2017-01-01’;

++

| count(*) |

++

|  5660651 |

++

1 row in set (16.56 sec)

 

mysql> select count(*) from ontime where flightdate > ‘2017-01-01’;

++

| count(*) |

++

|  5660651 |

++

1 row in set (16.36 sec)

As we can see the results are very stable. It does not use any cache (ie: innodb buffer pool) either. The result is also interesting: utilizing multiple threads (up to 16 threads) and reading data from disk (using disk cache, probably) can be ~10x faster compared to reading from memory in a single thread.

Result: ~10x performance gain, no index used

Query 1b: simple, avg

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

set aurora_pq = 1; set aurora_pq_force=1;

 

select avg(DepDelay) from ontime;

++

| avg(DepDelay) |

++

|        8.2666 |

++

1 row in set (1 min 48.17 sec)

 

 

set aurora_pq = 0; set aurora_pq_force=0;  

 

select avg(DepDelay) from ontime;

++

| avg(DepDelay) |

++

|        8.2666 |

++

 

1 row in set (2 min 49.95 sec)

 

Here we can see that PQ gives use ~2x performance increase.

Summary of simple query performance

Here is what we learned comparing Aurora PQ performance to native MySQL query execution:

  1. Select count(*), not using index: 10x performance increase with Aurora PQ.
  2. select avg(…), not using index: 2x performance increase with Aurora PQ.

Query 2: Complex filter, single table

The following query will always be slow in MySQL. This combination of the filters in the WHERE condition makes it extremely hard to prepare a good set of indexes to make this query faster.

select SQL_CALC_FOUND_ROWS

FlightDate, UniqueCarrier as carrier, FlightNum, Origin, Dest

FROM ontime

WHERE

  DestState not in (‘AK’, ‘HI’, ‘PR’, ‘VI’)

  and OriginState not in (‘AK’, ‘HI’, ‘PR’, ‘VI’)

  and flightdate > ‘2015-01-01’

  and ArrDelay < 15

and cancelled = 0

and Diverted = 0

and DivAirportLandings = ‘0’

ORDER by DepDelay DESC

LIMIT 10;

Let’s compare the query performance with and without PQ.

PQ disabled:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

mysql> set aurora_pq_force = 0;

Query OK, 0 rows affected (0.00 sec)

mysql> set aurora_pq = 0;                                                                                                                                                                  

Query OK, 0 rows affected (0.00 sec)

 

mysql> explain select SQL_CALC_FOUND_ROWS FlightDate, UniqueCarrier as carrier, FlightNum, Origin, Dest FROM ontime WHERE    DestState not in (‘AK’, ‘HI’, ‘PR’, ‘VI’) and OriginState not in (‘AK’, ‘HI’, ‘PR’, ‘VI’) and flightdate > ‘2015-01-01’     and ArrDelay < 15 and cancelled = 0 and Diverted = 0 and DivAirportLandings = 0 ORDER by DepDelay DESC LIMIT 10\G

*************************** 1. row ***************************

          id: 1

 select_type: SIMPLE

       table: ontime

        type: ALL

possible_keys: NULL

         key: NULL

     key_len: NULL

         ref: NULL

        rows: 173706586

       Extra: Using where; Using filesort

1 row in set (0.00 sec)

 

mysql> select SQL_CALC_FOUND_ROWS FlightDate, UniqueCarrier as carrier, FlightNum, Origin, Dest FROM ontime WHERE    DestState not in (‘AK’, ‘HI’, ‘PR’, ‘VI’) and OriginState not in (‘AK’, ‘HI’, ‘PR’, ‘VI’) and flightdate > ‘2015-01-01’     and ArrDelay < 15 and cancelled = 0 and Diverted = 0 and DivAirportLandings = 0 ORDER by DepDelay DESC LIMIT 10;

++++++

| FlightDate | carrier | FlightNum | Origin | Dest |

++++++

| 20171009 | OO      | 5028      | SBP    | SFO  |

| 20151103 | VX      | 969       | SAN    | SFO  |

| 20150529 | VX      | 720       | TUL    | AUS  |

| 20160311 | UA      | 380       | SFO    | BOS  |

| 20160613 | DL      | 2066      | JFK    | SAN  |

| 20161114 | UA      | 1600      | EWR    | LAX  |

| 20161109 | WN      | 2318      | BDL    | LAS  |

| 20161109 | UA      | 1652      | IAD    | LAX  |

| 20161113 | AA      | 23        | JFK    | LAX  |

| 20161112 | UA      | 800       | EWR    | SFO  |

++++++

10 rows in set (3 min 42.47 sec)

/* another run */

10 rows in set (3 min 46.90 sec)

This query is 100% cached. Here is the graph from PMM showing the number of read requests:

  1. Read requests: logical requests from the buffer pool
  2. Disk reads: physical requests from disk

Buffer pool requests:

Buffer pool requests from PMM

Now let’s enable and force PQ:

PQ enabled:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

mysql> set session aurora_pq = 1;

Query OK, 0 rows affected (0.00 sec)

 

mysql> set aurora_pq_force = 1;                                                                                                                              Query OK, 0 rows affected (0.00 sec)

 

mysql> explain select SQL_CALC_FOUND_ROWS FlightDate, UniqueCarrier as carrier, FlightNum, Origin, Dest FROM ontime WHERE    DestState not in (‘AK’, ‘HI’, ‘PR’, ‘VI’) and OriginState not in (‘AK’, ‘HI’, ‘PR’, ‘VI’) and flightdate > ‘2015-01-01’     and ArrDelay < 15 and cancelled = 0 and Diverted = 0 and DivAirportLandings = 0 ORDER by DepDelay DESC LIMIT 10\G

*************************** 1. row ***************************

          id: 1

 select_type: SIMPLE

       table: ontime

        type: ALL

possible_keys: NULL

         key: NULL

     key_len: NULL

         ref: NULL

        rows: 173706586

       Extra: Using where; Using filesort; Using parallel query (12 columns, 4 filters, 3 exprs; 0 extra)

1 row in set (0.00 sec)

 

 

mysql> select SQL_CALC_FOUND_ROWS                                                                                                                                                                      -> FlightDate, UniqueCarrier as carrier, FlightNum, Origin, Dest -> FROM ontime

   -> WHERE

   ->  DestState not in (‘AK’, ‘HI’, ‘PR’, ‘VI’)

   ->  and OriginState not in (‘AK’, ‘HI’, ‘PR’, ‘VI’)

   ->  and flightdate > ‘2015-01-01’

   ->   and ArrDelay < 15

   -> and cancelled = 0

   -> and Diverted = 0

   -> and DivAirportLandings = 0

   ->  ORDER by DepDelay DESC

   -> LIMIT 10;

++++++

| FlightDate | carrier | FlightNum | Origin | Dest |

++++++

| 20171009 | OO      | 5028      | SBP    | SFO  |

| 20151103 | VX      | 969       | SAN    | SFO  |

| 20150529 | VX      | 720       | TUL    | AUS  |

| 20160311 | UA      | 380       | SFO    | BOS  |

| 20160613 | DL      | 2066      | JFK    | SAN  |

| 20161114 | UA      | 1600      | EWR    | LAX  |

| 20161109 | WN      | 2318      | BDL    | LAS  |

| 20161109 | UA      | 1652      | IAD    | LAX  |

| 20161113 | AA      | 23        | JFK    | LAX  |

| 20161112 | UA      | 800       | EWR    | SFO  |

++++++

10 rows in set (41.88 sec)

/* run 2 */

10 rows in set (28.49 sec)

/* run 3 */

10 rows in set (29.60 sec)

Now let’s compare the requests:

InnoDB Buffer Pool Requests

As we can see, Aurora PQ is almost NOT utilizing the buffer pool (there are a minor number of read requests. Compare the max of 4K requests per second with PQ to the constant 600K requests per second in the previous graph).

Result: ~8x performance gain

Query 3: Complex filter, join “reference” table

In this example I join two tables: the main “ontime” table and a reference table. If we have both tables without indexes it will simply be too slow in MySQL. To make it better, I have created an index for both tables and so it will use indexes for the join:

CREATE TABLE `carriers` (

 `carrier_code` varchar(8) NOT NULL DEFAULT ,

 `carrier_name` varchar(200) DEFAULT NULL,

 PRIMARY KEY (`carrier_code`),

 KEY `carrier_name` (`carrier_name`)

) ENGINE=InnoDB DEFAULT CHARSET=latin1

 

mysql> show create table ontime_ind\G

...

 PRIMARY KEY (`id`),

 KEY `comb1` (`Carrier`,`Year`,`ArrDelayMinutes`),

 KEY `FlightDate` (`FlightDate`)

) ENGINE=InnoDB AUTO_INCREMENT=178116912 DEFAULT CHARSET=latin1

Query:

select SQL_CALC_FOUND_ROWS

FlightDate, UniqueCarrier, TailNum, FlightNum, Origin, OriginCityName, Dest, DestCityName, DepDelay, ArrDelay

FROM ontime_ind o

JOIN carriers c on o.carrier = c.carrier_code

WHERE

  (carrier_name like ‘United%’ or carrier_name like ‘Delta%’)

  and ArrDelay > 30

  ORDER by DepDelay DESC

LIMIT 10\G

PQ disabled, explain plan:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

mysql> set aurora_pq_force = 0;

Query OK, 0 rows affected (0.00 sec)

 

mysql> set aurora_pq = 0;                                                                                                                                                                  

Query OK, 0 rows affected (0.00 sec)

 

mysql> explain

   -> select SQL_CALC_FOUND_ROWS

   -> FlightDate, UniqueCarrier, TailNum, FlightNum, Origin, OriginCityName, Dest, DestCityName, DepDelay, ArrDelay

   -> FROM ontime_ind o

   -> JOIN carriers c on o.carrier = c.carrier_code

   -> WHERE

   ->  (carrier_name like ‘United%’ or carrier_name like ‘Delta%’)

   ->  and ArrDelay > 30

   ->  ORDER by DepDelay DESC

   -> LIMIT 10\G

*************************** 1. row ***************************

          id: 1

 select_type: SIMPLE

       table: c

        type: range

possible_keys: PRIMARY,carrier_name

         key: carrier_name

     key_len: 203

         ref: NULL

        rows: 3

       Extra: Using where; Using index; Using temporary; Using filesort

*************************** 2. row ***************************

          id: 1

 select_type: SIMPLE

       table: o

        type: ref

possible_keys: comb1

         key: comb1

     key_len: 3

         ref: ontime.c.carrier_code

        rows: 2711597

       Extra: Using index condition; Using where

2 rows in set (0.01 sec)

As we can see MySQL uses indexes for the join. Response times:

/* run 1 – cold run */

10 rows in set (29 min 17.39 sec)

/* run 2  – warm run */

10 rows in set (2 min 45.16 sec)

PQ enabled, explain plan:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

mysql> explain

   -> select SQL_CALC_FOUND_ROWS

   -> FlightDate, UniqueCarrier, TailNum, FlightNum, Origin, OriginCityName, Dest, DestCityName, DepDelay, ArrDelay

   -> FROM ontime_ind o

   -> JOIN carriers c on o.carrier = c.carrier_code

   -> WHERE

   ->  (carrier_name like ‘United%’ or carrier_name like ‘Delta%’)

   ->  and ArrDelay > 30

   ->  ORDER by DepDelay DESC

   -> LIMIT 10\G

*************************** 1. row ***************************

          id: 1

select_type: SIMPLE

       table: c

        type: ALL

possible_keys: PRIMARY,carrier_name

         key: NULL

     key_len: NULL

         ref: NULL

        rows: 1650

       Extra: Using where; Using temporary; Using filesort; Using parallel query (2 columns, 0 filters, 1 exprs; 0 extra)

*************************** 2. row ***************************

          id: 1

select_type: SIMPLE

       table: o

        type: ALL

possible_keys: comb1

         key: NULL

     key_len: NULL

         ref: NULL

        rows: 173542245

       Extra: Using where; Using join buffer (Hash Join Outer table o); Using parallel query (11 columns, 1 filters, 1 exprs; 0 extra)

2 rows in set (0.00 sec)

As we can see, Aurora does not use any indexes and uses a parallel scan instead.

Response time:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

mysql> select SQL_CALC_FOUND_ROWS

   -> FlightDate, UniqueCarrier, TailNum, FlightNum, Origin, OriginCityName, Dest, DestCityName, DepDelay, ArrDelay

   -> FROM ontime_ind o

   -> JOIN carriers c on o.carrier = c.carrier_code

   -> WHERE

   ->  (carrier_name like ‘United%’ or carrier_name like ‘Delta%’)

   ->  and ArrDelay > 30

   ->  ORDER by DepDelay DESC

   -> LIMIT 10\G

...

 

*************************** 4. row ***************************

 

   FlightDate: 20170504

UniqueCarrier: UA

      TailNum: N68821

    FlightNum: 1205

       Origin: KOA

OriginCityName: Kona, HI

         Dest: LAX

 DestCityName: Los Angeles, CA

     DepDelay: 1457

     ArrDelay: 1459

*************************** 5. row ***************************

   FlightDate: 19910312

UniqueCarrier: DL

      TailNum:

    FlightNum: 1118

       Origin: ATL

OriginCityName: Atlanta, GA

         Dest: STL

 DestCityName: St. Louis, MO

...

 

10 rows in set (28.78 sec)

 

 

mysql> select found_rows();

 

++

| found_rows() |

++

|      4180974 |

++

 

1 row in set (0.00 sec)

Result: ~5x performance gain

(this is actually comparing the index cached read to a non-index PQ execution)

Summary

Aurora PQ can significantly improve the performance of reporting queries as such queries may be extremely hard to optimize in MySQL, even when using indexes. With indexes, Aurora PQ response time can be 5x-10x better compared to the non-parallel, fully cached operations. Aurora PQ can help improve performance of complex queries by performing parallel reads.

The following table summarizes the query response times:

Query Time, No PQ, index Time, PQ
select count(*) from ontime where flightdate > ‘2017-01-01’ 2 min 48.81 sec 16.53 sec
select avg(DepDelay) from ontime; 2 min 49.95 sec 1 min 48.17 sec
select SQL_CALC_FOUND_ROWS

FlightDate, UniqueCarrier as carrier, FlightNum, Origin, Dest

FROM ontime

WHERE

DestState not in (‘AK’, ‘HI’, ‘PR’, ‘VI’)

and OriginState not in (‘AK’, ‘HI’, ‘PR’, ‘VI’)

and flightdate > ‘2015-01-01’

and ArrDelay < 15

and cancelled = 0

and Diverted = 0

and DivAirportLandings = 0

ORDER by DepDelay DESC

LIMIT 10;

3 min 42.47 sec 28.49 sec
select SQL_CALC_FOUND_ROWS

FlightDate, UniqueCarrier, TailNum, FlightNum, Origin, OriginCityName, Dest, DestCityName, DepDelay, ArrDelay

FROM ontime_ind o

JOIN carriers c on o.carrier = c.carrier_code

WHERE

(carrier_name like ‘United%’ or carrier_name like ‘Delta%’)

and ArrDelay > 30

ORDER by DepDelay DESC

LIMIT 10\G

2 min 45.16 sec 28.78 sec


Photo by Thomas Lipke on Unsplash

Related

via MySQL Performance Blog
Using Parallel Query with Amazon Aurora for MySQL

AWS launches Backup, a fully-managed backup service for AWS

Amazon’s AWS cloud computing service today launched Backup, a new tool that makes it easier for developers on the platform to back up their data from various AWS services and their on-premises apps. Out of the box, the service, which is now available to all developers, lets you set up backup policies for services like Amazon EBS volumes, RDS databases, DynamoDB tables, EFS file systems and AWS Storage Gateway volumes. Support for more services is planned, too. To back up on-premises data, businesses can use the AWS Storage Gateway.

The service allows users to define their various backup policies and retention periods, including the ability to move backups to cold storage (for EFS data) or delete them completely after a certain time. By default, the data is stored in Amazon S3 buckets.

Most of the supported services, except for EFS file systems, already feature the ability to create snapshots. Backup essentially automates that process and creates rules around it, so it’s no surprise that the pricing for Backup is the same as for using those snapshot features (with the exception of the file system backup, which will have a per-GB charge). It’s worth noting that you’ll also pay a per-GB fee for restoring data from EFS file systems and DynamoDB backups.

Currently, Backup’s scope is limited to a given AWS region, but the company says that it plans to offer cross-region functionality later this year.

“As the cloud has become the default choice for customers of all sizes, it has attracted two distinct types of builders,” writes Bill Vass, AWS’s VP of Storage, Automation, and Management Services. “Some are tinkerers who want to tweak and fine-tunee the full range of AWS services into a desired architecture, and other builders are drawn to the same breadth and depth of functionality in AWS, but are willing to trade some of the service granularity to start at a higher abstraction layer, so they can build even faster. We designed AWS Backup for this second type of builder who has told us that they want one place to go for backups versus having to do it across multiple, individual services.”

Early adopters of AWS Backup are State Street Corporation, Smile Brands and Rackspace, though this is surely a service that will attract its fair share of users as it makes the life of admins quite a bit easier. AWS does have quite a few backup and storage partners, though, who may not be all that excited to see AWS jump into this market, too, though they often offer a wider range of functionality — including cross-region and offsite backups — than AWS’s service.

 


via TechCrunch
AWS launches Backup, a fully-managed backup service for AWS

27 Free Alternatives to Adobe’s Expensive App Subscriptions


Adobe appears to have upset a number of users with another price increase for its app subscriptions. While the hit only appears to be targeting specific countries at this point—you’re spared, North American users—there’s no reason to think that you won’t have to pay more to subscribe to an Adobe app (or its whole suite of creative apps) at some future point. That’s business, folks.

As you can imagine, Adobe’s price increase has set off a flurry of activity on the internet, with many annoyed users jumping onto Twitter threads and blog posts to suggest alternatives to Adobe’s ever-more-expensive subscription apps.

I ran through @burgerdrome’s Twitter thread, as well as an excellent software-recommendations thread started by @TubOfCoolWhip and this handy image of recommendations from “Cullen,” who I would link to if I knew who they were. From there, I created this list of 27 good alternatives to Adobe’s Creative Cloud apps based on what people appeared to be excited about (or recommend in droves).

I haven’t tried out all of these apps myself, nor am I the target audience for them—as I don’t really dabble in 3D animation, alas. While we normally recommend apps we’ve used at Lifehacker, in this case, I’ve included recommendations from the various Twitter users who have suggested them when applicable. (It’s tough, as some apps just got called out by name, which is great for making a list, but not very helpful when describing an app’s features.)

If you don’t like any of these picks, you can always try befriending an educator (or a student) to score that sweet $20/month pricing for Adobe’s full subscription. A word of caution, however: That only works for the first year. After that, you’ll get charged the full, standard rate.

Apps for painting, graphic design, or photo editing

Krita (Windows / Mac)

“I can personally recommend Krita as a viable open illustration program. On the commercial side, I’ve heard good things of Clip Studio Paint and Paint Tool SAI. Krita also has re-editable file layers, filter/effect layers and layer styles.”@AwrySquare

Sketchbook (Windows / Mac)

“I use Sketchbook with my pen display and I can recommend it. It has a decently easy-to-navigate UI and allows you to save in a .psd format for an easy transfer. The only thing it really needs is clipping groups.”@xx_unsung_xx

MediBang Paint (Windows / Mac / Mobile)

“a really good free ipad app for art is Medibang paint. It’s just as simple to use as procreate, and has all the features and more :)!!!”@1lonelyegg

Paint.net (Windows)

“Getpaint.net is a great free Photoshop alternative and @inkscape is a great free Illustrator alternative. Been using those for years, and I have all the Adobe products, but those are still my go to’s. I basically only use my Adobe subscription for Premiere and AE.”@alexcchichester

Pixlr X (Web)

“Pixlr is a personal favorite! I believe they just added a paid option to get rid of ads but there’s still a fully functioning free version”@notjoykeenan

GIMP (Windows / Mac)

“For photo editing,GIMP is pretty much Photoshop but free ! While the UI may be less user-friendly,it can give out nice results !” @FarowlHd

Photoscape X (Windows / Mac)

“Photoscape is free and provides a pretty basic photo editing software! you can do a lot with it like make gifs and batch-edit photos in addition to your basics. been using it for 5+ years and have rarely needed something more” @trisk_philia

FireAlpaca (Windows / Mac)

“well now im glad i stick to sai and firealpaca. at least they arent laggy as shit and confusing to look at” @finnifinite

If you need a little more than that to consider FireAlpaca for your setup, the app comes with plenty of standard and quirky brushes for digitally painting your next great masterpiece (or comic). You can even make your own, if you’re feeling especially creative. For those looking to draw some comics, built-in templates make it easy to create specific layouts for a strip. The app’s “Onion Skin” mode also makes it easy to draw animations, as you’ll be creating new layers, or frames, while viewing the previous frame as a reference point.


Apps for creating storyboards

Storyboarder (Windows / Mac)

“The features on this are pretty good, and you DO NOT have to be able to render/draw well to use this! It can create shot types from key words, which is…. wild. :o” @TubOfCoolWhip


Apps for editing videos or creating video effects

HitFilm Express (Windows / Mac)

“Somone may have already mentioned these two but VSDC Editor and Hitflim are neat free editing softwares.”@NotQueenly

If I’m correct, Hitfilm Express an excellent tool for creating special effects—much more so than your standard video editing app, which might not be quite as fully featured for this kind of work. If you’re just looking to edit and trim videos, and maybe add a simple text overlay, other video apps on this list might be a better fit.

Shotcut (Windows / Mac)

“I found [Shotcut] to be a very good free editor for video editing. It’s worked very well for me and i still use it for smaller things.”@Monkeygameal

If you’re trying to get crazy, like edit 360-degree videos—as PCMag notes—this might not be the app for you. But for basic video editing with a reasonably uncluttered interface, you can’t go wrong with this free app.

DaVinci Resolve (Windows / Mac)

“Because of Davinci resolve I only have the photoshop/ light room bundle. Once I can find a better alternative to photoshop and light room. Im going to ditch that too.” @Breonnick_5

Kdenlive (Windows / Mac—sort-of)

Although this multi-track video editor is mainly for Linux users, you’ll still find some slightly older Windows and Mac builds to experiment with. Since the app uses FFmpeg libraries, you can import any video or audio file you want—pretty much. You also get a healthy number of transformations and effects to play with, which you can keyframe for greater precision.


Apps for 3D modeling, animation, or vector graphics

Blender (Windows / Mac)

“I hate Maya for similar reasons and stick to blender whenever I can.”@IRBlayne

Blender is the big-guns 3D modeling tool that you dabble with when you don’t want to pay for something like 3DS or Maya. The learning curb is steep, but it’s worth mastering if you’re serious about exploring the space. Once you get good, you can do a lot of amazing things with this free app:

Lumion (Windows, free for students)

“If you are a student, the student version of Lumion is FREE. It is an architecture program that renders reeaal fast and does all kinds of neat stuff such as automatic sites, insertable animations of people doing stuff, you can set things on fire, weather settings, and more.”@samanthagiford8

Synfig (Windows / Mac)

“Synfig for animation! it’s vector-based and works similar to Flash, it can’t do interactive stuff but Flash games are kind of dying anyway”@ljamesart

Anything that’s similar to Adobe Flash, but isn’t Adobe Flash, is a win in our book.

SketchUp (Web)

You’ll find this recommendation on the aforementioned “Cullen” list, which indicates it’s a great program for basic 3D modeling. Since it’s (now) completely web-based, you can use it right in your favorite browser on Windows or Mac—or on a Chromebook, I suppose. And, yes, everything you do automatically saves to the cloud, don’t worry.

MagicaVoxel (Windows / Mac)

Here’s another entry on the “Cullen” list—this time, their recommendation for a voxel/brick 3D modeling program. I’m not much of an artist, nor am I a Minecraft wizard (but I do love amazing pixel art), so I’ll instead leave you with a comment from this inspiring 2015 blog post: “I started with [MagicaVoxel]5 months ago and feel like I have really mastered the tool. I saw a Tweet of voxel art image made on Magica Voxel from Ephtracy. That was when I just finished Monument Valley, which I loved. I had to try that tool and fell in love with it right away.”

MakeHuman (Windows / Mac) 

The mysterious “Cullen” also recommends MakeHuman if you want to fiddle around with creating digital characters in three dimensions. If I’m correct, you can import your creations into another app on our list—Blender—to animate them, which is as close as you’ll get to full-featured rendering software like 3DS or Maya without plunking down a ton of change.

Inkscape (Windows / Mac)

“The vector program Inkscape is a wonderful free alternative to Adobe Illustrator”@GrimdorkDesign

I consistently see Inkscape mentioned as an alternative to Adobe Illustrator around the web. I don’t use Illustrator myself, but if I did, this would be the first app I installed to escape Adobe’s subscription fees.


Apps for editing audio and creating music

LMMS (Windows / Mac)

“If we’re including music/audio editing software, LMMS and Cakewalk by BandLab are both good free DAWs!”@MystSaphyr

“DAWs,” for those not in the know, is short for “Digital Audio Workstations.” If you’re making music, go with LMMS (or Cakewalk, below.) If you need to cut audio or convert something to an MP3, you’ll want an app like Audacity.

Cakewalk (Windows)

(See previous recommendation. Thanks, @MystSaphyr!)

ocenaudio (Windows / Mac)

“I’d like to add Ocean Audio [sic] as a simple audio editor as well as REAPER as an inexpensive & extremely powerful DAW (with infinite trial period)“ @fuzzblob

Audacity (Windows / Mac)

I almost shouldn’t need to say anything about Audacity at this point, as it’s been one of the best free audio editors around for years. It’s my go-to app whenever I need to cut and rearrange audio super-quick.


Apps for desktop publishing

Scribus (Windows / Mac)

In response to a question about InDesign alternatives: “Affinity has one coming/out already. But yeah, only other thing I’ve found is Scribus.” @dukiswa

Canva (Web)

“Pixlr was a really great place to start as an alternative to Photoshop and Canva works well as an alternative to InDesign !!”@lexgts


via Gizmodo
27 Free Alternatives to Adobe’s Expensive App Subscriptions