The Trillion Fold Increase In Computing Power, Visualized

The Trillion Fold Increase In Computing Power, Visualized

It’s easy to get hung up over the imperfections in our technology (srsly Apple, is it that hard to give a phone a back button?) and forget just how astounding modern processing power is. A community of IT professionals called Experts Exchange has now produced a fascinating infographic to remind us.

Top image: Watch as the size of hard disk drives shrinks while storage capacity increases from 1956 to 2015.

The visualization below, inspired by the recent 50th anniversary of Moore’s law, tells the story of the trillion fold increase in computing performance we’ve witnessed over the past sixty years. That’s impressive enough, but some of the other finds are downright astounding. The Apollo guidance computer that took early astronauts to the moon, for instance, has the processing power of 2 Nintendo Entertainment Systems, while the Cray-2 supercomputer from 1985—the fastest machine in the world for its time—roughly measures up to an iPhone 4.

Plenty of interesting insights to be found here. You can check out Expert Exchange’s original post for more info on their sources and methodology.

The Trillion Fold Increase In Computing Power, Visualized


Follow Maddie on Twitter or contact her at maddie.stone@gizmodo.com

via Gizmodo
The Trillion Fold Increase In Computing Power, Visualized

Here’s A Detailed Walk Through Of How JPEG Compression Works

Understanding the science behind JPEG compression might not be on everyone’s agenda. In fact, it probably sounds incredibly boring, if not complicated to a lot of folks. But, in this 12-minute long clip, image analyst Mike Pound guides you through the entire process as he explains the magic of Discrete Cosine Transform (DCT)–using language that’s easy to […]

The post Here’s A Detailed Walk Through Of How JPEG Compression Works appeared first on DIY Photography.


via DIYPhotography.net -Hacking Photography, One Picture At A Time
Here’s A Detailed Walk Through Of How JPEG Compression Works

To-Do List: Apply For The TC Pitch-Offs In Austin And Seattle!

14394608374_090d4cd343_k It’s shaping up to be one amazing summer, and the week we’re most excited about here at TechCrunch is the week we hit up the wonderful cities of Austin and Seattle, on June 23 and June 25 respectively. We’ll be hosting the world-renowned TC Pitch-Off in both cities, selecting around ten startups to pitch their products in sixty seconds or less to a panel of local VCs and… Read More


via TechCrunch
To-Do List: Apply For The TC Pitch-Offs In Austin And Seattle!

Top 10 Smart Ways to Organize and Upgrade Your Garage

Top 10 Smart Ways to Organize and Upgrade Your Garage

The garage is often a neglected, messy space. With a little organization and a few handy projects, though, we can get much more use out of our garages and also modernize them. Here are our top 10 garage upgrade ideas.

10. Install a Ceiling Storage System

Top 10 Smart Ways to Organize and Upgrade Your Garage

Make better use of your garage’s space and get things off the floor with this DIY project. It costs about $75 for all of the materials and is a great way to store seldom-used things like holiday decorations.

9. Store Garden Tools and Other Supplies Smartly on the Wall

Top 10 Smart Ways to Organize and Upgrade Your Garage

With just two boards and a saw you can make this simple tool rack for your wall. For smaller tools, there’s always the trusty pegboard (and alternatives.) If you have a lot of tools, perhaps a hinged pegboard system is for you or this rolling pegboard caddy.

8. Make a Wall-Mounted, Folding Workbench

Top 10 Smart Ways to Organize and Upgrade Your Garage

If your garage is tight on space but you still want to use it as your workspace, this DIY wall-mounted foldable workbench not only stores your tools but also provides project space on an as-needed basis.

7. Upgrade Your Garage Workshop with Simple DIY Projects

Top 10 Smart Ways to Organize and Upgrade Your Garage

You don’t have to spend a fortune to improve your garage workshop. A sheet metal workbench cover, PVC drawer organizers, and custom storage pieces could go a long way towards making your garage a nicer place to DIY. You can also build your own garage storage shelves for under $50 or turn an old file cabinet into a garden tools storage unit.

6. Create an Entire Storage Wall System

Top 10 Smart Ways to Organize and Upgrade Your Garage

Or perhaps go whole hog and turn all of your wall space into a storage system to hold everything from your tools to your garden hose and ladder. The best part of this project is you can customize it exactly for your storage needs.

5. Make Parking in the Garage Easier with Clever Uses

Top 10 Smart Ways to Organize and Upgrade Your Garage

It only takes one experience knocking over a bike or banging the car door into the wall to do some damage. The versatile pool noodle can protect your car door from your wall. You can also make your own $10 garage parking bumper so you’ll never drive too far into the garage. Alternately, hang a tennis ball from your garage ceiling so you’ll always pull in to just the right spot.

4. Use Your Garage for More Than Just Storing Stuff

Top 10 Smart Ways to Organize and Upgrade Your Garage

If you’ve got room in your garage and somewhere else to park the car, consider converting that space into more than just a storage space. The photo above shows the office half of a 2-car garage. Offices set in garages can look surprisingly like a regular home office. If you don’t need office space, perhaps you’d like to turn your garage into a home gym.

3. Monitor Your Garage Door’s Opened or Closed Status

Top 10 Smart Ways to Organize and Upgrade Your Garage

Forgetting to close the garage door can be dangerous, when thieves and animals are on the prowl. Make your own garage door indicator light so you can see with a glance from indoors if your garage door is open. Alternatively, you can build an open an open garage alert system with an old RC car and controller or monitor your door’s status with your phone and a sensor system.

2. Open Your Garage Door with Your Phone

When you do want your garage door open, your phone can be the remote. Siri and a Raspberry Pi make the magic happen for iPhone users, while a Bluetooth board and a free app do the job on Android. (The latter project also remotely starts your car with your phone!)

1. Organize Your Garage into Zones

Top 10 Smart Ways to Organize and Upgrade Your Garage

You likely have multiple uses for your garage and a variety of things to store there. Divide that space into six distinct zones and it’ll be easier to find things and put them away faster. And maybe even have room in your garage for your car.

Illustration by Tina Mailhot-Roberge.


Lifehacker’s Weekend Roundup gathers our best guides, explainers, and other posts on a certain subject so you can tackle big projects with ease. For more, check out our Weekend Roundup and Top 10 tags.


via Lifehacker
Top 10 Smart Ways to Organize and Upgrade Your Garage

The Groundbreaking History Of Star Wars Toys

The Groundbreaking History Of Star Wars Toys

When Star Wars released in 1977, the face of science fiction in popular culture was changed forever — but a year later, the movie helped transform the toy industry as well. Since then, Star Wars and the toys it inspired have been forever linked, a story that can just as easily be told through figures as it can the films.

Top image from the cover of Stephen J. Sansweet’s Star Wars: The Ultimate Action Figure Collection.

An Unexpected Alliance

When George Lucas and 20th Century Fox were trying to market Star Wars, they planned for something almost entirely unprecedented at the time — a marketing deluge, and a full scale licensing project that would see t-shirts, posters, lunchboxes and yes, toys, covered in the movie’s characters, hit shelves. In a move that, in hindsight, was incredibly shrewd, Lucas negotiated with Fox to take the bulk of revenue from merchandise sales, with neither side believing that the movie’s tepid response before release could lead to much in terms of profit.

The lukewarm reaction spread to licensees too. Lucasfilm and Fox first offered the Mego Corporation — whose 8” licensed dolls of DC superheroes, Star Trek and more had made them one of the most powerful toy makers of the 1970’s — the deal to create Star Wars dolls, but the company passed, unimpressed by the movie. After attempting to shop the license around to other toy makers, in 1976 it fell to Kenner, then a subsidiary of General Mills. Kenner President Bernie Loomis saw an opportunity to make good toys with the license (especially in the then relatively new space of 3.75” scaled action figures, cheaper to produce than the larger toys), but expected Star Wars to be a fleeting venture for the company.

Little did anyone involved know how wrong they would be.

The Early Bird Gets The Gift Certificate

The Groundbreaking History Of Star Wars Toys

Star Wars released in May 1977 to rapturous approval, becoming an overnight sensation — and kids didn’t just want to see the movie; they wanted toys. Kenner were caught flat-footed at the demand, finding that they wouldn’t even have figures out for the lucrative Christmas period of that year. To do nothing would have meant losing out on millions of dollars.

So they made a decision that was, by all accounts at the time, completely ludicrous: They sold people an empty box. The Early Bird Certificate was a box containing a cardboard display stand featuring the characters from the film, stickers, and a certificate for kids to mail away to Kenner to receive four figures in 1978: Luke Skywalker, R2-D2, Princess Leia and Chewbacca. The box was savaged by the media, and although sales were poor, the move kept Star Wars figures in the public’s mind, ready for their 1978 release.

Your First Step Into A Larger World

The Groundbreaking History Of Star Wars Toys

When the Star Wars line first hit in 1978, any damage that criticism of the Early Bird Certificate could have done was wiped out almost instantaneously. Joined by another eight figures, and with playsets and vehicles following later in the year, Kenner’s toyline was a smash success, making over $100 million in its first year alone — with demand often outstripping supply. Kenner’s toy line became the icon of the new era of 3.75” figures.

The following year, the company capitalised on the announcement of a sequel, The Empire Strikes Back, with another mail-away campaign: one that proved to be far more controversial than the Early Bird Cerificate.

In the new promotion, kids could mail four proofs of purchase from any Star Wars figure and get a sneak peek of a new Empire toy, the mysterious bounty hunter Boba Fett, who had made his first appearance the year prior in the infamously atrocious Star Wars Holiday Special. The figure came with a heavily advertised rocket-firing jetpack feature, but shortly before Boba Fett went to market, a rash of health-and-safety fears caused Kenner to make a late decision to glue the rocket into the backpack securely. Kenner has always maintained that they never released a rocket-firing Fett into the wild, but several such figures (as well as early production prototypes) have made their way into the hands of collectors over the years, making it one of the most valuable Star Wars toys ever made, selling for upwards of $2000 when one appears at auction.

By the release of Return of the Jedi, the line had expanded to contain 79 figures, with oodles of playsets, vehicles and creatures released. But without movies to support them, sales slowly began to dwindle. Kenner attempted to offset the decline with brief lines based on the animated Droids and Ewok cartoons, the first non-movie Star Wars toys ever made, but it was too late. In 1985, after 250 million Star Wars figures had been shipped over the world, Kenner ended the toyline. Plans were even made for a spinoff line the following year, The Epic Continues, featuring new characters based on a storyline created by Kenner in an attempt to reignite interest in the toys, but Lucasfilm rejected the move. For now, Star Wars as a toyline was over.

A New Hope

The Groundbreaking History Of Star Wars Toys

But it only took a decade for that to all change. By the mid 1990s, George Lucas had announced his intent to create three brand new Star Wars films, prequel movies to the originals, and specially remastered editions of the classic trilogy were being prepared to hit cinemas once again. Kenner, now owned by Hasbro, decided to capitalise on the excitement surrounding Star Wars by going back and creating new figures, a spiritual sequel to the Power of the Force Line that ended the original toyline in 1985.

The figures were a massive success, but collectors and fans of the original figures were appalled. The initial Hasbro/Kenner toys — published under the Kenner name rather than Hasbro to capitalise on nostalgia — only included one extra piece of articulation (they could turn at the waist), but more egregious was what Hasbro had dubbed the “Hero Age” sculpts. The initial figures were muscled up to the point where they were unrecognisable (even Princess Leia, whose figure was quickly dubbed “Monkey Leia” by fans for her awkward pose and poor face sculpt) and with a weirdly wide-legged stance that made it impossible to get the figures to properly sit in vehicles and ships.

But nevertheless, the new figures were immensely popular, spurred by the release of the Special Editions. Hasbro upped production on the Star Wars line once more, and even began incorporating elements of the then-fledgling Expanded Universe. Characters from the original Kenner line like Yak Face and Hammerhead were given their EU-sourced names (Saelt-Marae and Momaw Nadon, respectively), and for the first time, characters from the novels and video games were turned into toys, like Emperor’s Hand Mara Jade, or the bulky Dark Troopers from Star Wars: Dark Forces.

The Phantom Preview

The Groundbreaking History Of Star Wars Toys

But Hasbro’s focus on the original movies soon gave way to the insane anticipation for The Phantom Menace. This time, they opted not to go for a mail-away campaign. Instead, in an unprecedented move, they sold two new toys in 1998 to stoke excitement for the film: a figure of Samuel L. Jackson’s Mace Windu (complete with a blue lightsaber, ahead of the character getting a purple saber in Attack of the Clones) and a deluxe figure of a Battle Droid on a STAP speeder bike. Both figures came in special packaging, teasing the release of The Phantom Menace in 1999, and gave fans some of the first proper looks at new characters. They were lapped up over the year, and excitement for Phantom Menace toys reached fever pitch.

Aside from a surge in popularity due to the new movie, Hasbro decided to tie the toy line closer to the films by ditching the “Hero Age” sculpts, opting for a neutral and more naturally-bodied stance. The Phantom Menace figures also came with a voice chip accessory that, when used with a separate toy based on Jedi Master Qui-Gonn Jinn’s communicator from the film, played lines of dialogue from the films.

Attack Of The Super-Articulated Clones

The Groundbreaking History Of Star Wars Toys

But despite the success of the new toy line, and an invigorated public interest in Star Wars merchandise thanks to new movies coming out, Hasbro faced pressure to evolve the toy line even further. A common critique was the figures’ lack of articulation, something that had hardly changed since the original Kenner days, compared to other figures on the market — something Hasbro decided to rectify with their Attack of the Clones line in 2002. The first ever “super articulated” Star Wars toy, a Clone Trooper, went on sale in 2004, and was so popular the figure continued to be a part of Star Wars lines for the next half a decade.

Hasbro brought the increased articulation to a wider range of figures for its Revenge of the Sith toy line in 2005. Although excitement over the prequel saga had diminished since the late ‘90s, Hasbro made a huge push with what it thought could be the final years of the Star Wars toy line. As with The Phantom Menace toys before them, sneak preview figures were released (General Grievous, Utapaun politician Tion Medon, a Wookiee warrior and the R4-G9 astromech droid), but Hasbro also pushed midnight releases of the product line across US toy stores, encouraging fans to queue up in costume and celebrate the release of the toys.

Black And Blue

The Groundbreaking History Of Star Wars Toys

Despite the lack of a new movie to keep fans interested, the Star Wars toy line did not come to an end as Hasbro had feared. The success of the Revenge of the Sith toy line, mainly thanks to the improved sculpting and articulation it included, spurred Hasbro to go back to the previous films and make new figures that included the extra articulation and detail (they also added premium features like cloth clothing — earlier toys sculpted clothes out of plastic, and in the case of the first Kenner figures, vinyl sheets were used for cloaks and robes) they had previously lacked. The figure line would then be bolstered by the announcement of a new animated series, Star Wars: Clone Wars in 2008, but by and large Hasbro’s Star Wars line closed out the 2000s in relative normality.

However, times were changing in the toy industry. Action figure popularity was on the decline (in favour of an emphasis on construction toys, like Mega Bloks and Lego), and the rising price of oil had a severe impact on manufacturing costs. The heavily detailed and articulated figures that Star Wars had become known for were becoming too expensive to produce.

Instead of cancelling the line however, Hasbro made the decision to split it in two. For the first time in Star Wars toy history, figures would be produced in the 6” scale. Announced in 2013 and dubbed “The Black Series,” these figures would keep the high articulation and improved detailing and be aimed solely at the collectors market, with higher prices to match (prices doubling from the usual $10-12 to $20-25).

The Black Series, however, came at the cost of the long running 3.75” line. Now aimed solely at younger children instead of trying to balance between appealing to kids and diehard collectors, Hasbro decided to cut the super articulation and detailing introduced 8 years prior with Revenge of the Sith to keep costs down. Many fans bemoaned the end of an era, but with Hasbro’s Star Wars toys grossing nearly six billion dollars since 1995, there was plenty proof out there that there was still an audience for both kinds of toys.

The Merchandising Awakens

The Groundbreaking History Of Star Wars Toys

So where does that leave us in 2015? With a new era of Star Wars movie making upon us —following Disney’s acquisition of the Star Wars license and Episode VII on the way this December — toys will once again play an important role in how fans discover the new Star Wars universe. As it was with The Phantom Menace and Revenge of the Sith, the toys will be part of the earliest looks fans will see of the new movie (outside of teaser trailers, of course) when The Force Awakens merchandise is launched on September 4th, as part of a heavily marketed “Force Friday” celebration. Three months ahead of the film may sound unbelievable to most, but as we’ve seen in Star Wars’ own past, it’s rather restrained.

Even with nearly 40 years between the Early Bird Certificate and these new toys, the relationship between Star Wars and its action figures is stronger than ever.

via Gizmodo
The Groundbreaking History Of Star Wars Toys

Turn Burger Patties into Edible Bowls for the Ultimate Bunless Burger

Turn Burger Patties into Edible Bowls for the Ultimate Bunless Burger

Meet the “beer can burger,” a stuffed-with-everything burger that’s sure to be a crowd-pleaser at your next grilling party.

Savannah Sussman posted images of this burger work of art on Facebook. Take a big ball of ground beef and flatten the middle with a beer can to turn it into a bowl. Then you can add a slice of cheese at the bottom and fill the burger with mushrooms and onions or whatever you like. Sussman wraps the burger in bacon and tops it all off with more cheese on top. Grill for one hour at 300° F, she says, over indirect heat with the grill cover down.

This is definitely an indulgence, but also a neat way to have a burger with everything except the bun. Check out Sussman’s Facebook post for more photos.

Beer can burgers… | Facebook


via Lifehacker
Turn Burger Patties into Edible Bowls for the Ultimate Bunless Burger

Federal Circuit: Software is not Patent Eligible unless Claimed as a Process or Physical Object

In an interesting – though non-precedential – opinion, the Federal Circuit has ruled that a “speech-recognition interface” software lacks subject matter eligibility “because [the claims] are not directed to one of the four statutory categories of inventions identified in 35 U.S.C. § 101. The court writes: “[s]oftware may be patent eligible, but when a claim is not directed […]
via Patently-O » Patent
Federal Circuit: Software is not Patent Eligible unless Claimed as a Process or Physical Object

Watch the furthest flight ever flown on a real life hoverboard

Watch the furthest flight ever flown on a real life hoverboard

The Guinness World Records says that Catalina Alexandru Duru just pulled off the farthest flight ever traveled on a real life hoverboard: 905 feet and 2 inches. You can see him rise up 16 feet in the air on the hoverboard and then start cruising through the air over a lake with nothing but invisible underneath him in the video below.

The hoverboard Duru uses is more like a super powerful quadcopter-type hoverboard and not a hoverboard in the Back to the Future sense but it’s still pretty awesome. And also, the added bonus of not using BTTF-style? This one works over water.


SPLOID is delicious brain candy. Follow us on Facebook or Twitter.

via Gizmodo
Watch the furthest flight ever flown on a real life hoverboard

Internet Health Test Checks to See if Your ISP Slows Your Connection

Internet Health Test Checks to See if Your ISP Slows Your Connection

The FCC recently established new rules to prevent ISPs from selectively degrading your connection to certain services, like some have done in the past. This tool, from Fight for the Future, can check to see if this type of degradation is happening to you.

The tool checks your connection to different locations throughout the internet that are outside your ISPs own network. This helps determine whether there are choke points between various networks designed to slow down your connection. This type of slow down famously occurred when Comcast throttled Netflix in order to negotiate a higher fee.

On its own, the test can give you a decent idea of how consistent your connection speed is. However, the organization also puts the data from all users’ tests together to get a birds’ eye view of ISP activity. With enough people testing their connection, Fight for the Future hopes that we can see which companies are responsible for bottlenecks that violate net neutrality principles more easily.

Internet Health Test


via Lifehacker
Internet Health Test Checks to See if Your ISP Slows Your Connection

Creating and Restoring Database Backups With mysqldump and MySQL Enterprise Backup – Part 2 of 2

In part one of this post, I gave you a couple examples of how to backup your MySQL databases using mysqldump. In part two, I will show you how to use the MySQL Enterprise Backup (MEB) to create a full and partial backup.
MySQL Enterprise Backup provides enterprise-grade backup and recovery for MySQL. It delivers hot, online, non-blocking backups on multiple platforms including Linux, Windows, Mac & Solaris. To learn more, you may download a whitepaper on MEB.
MySQL Enterprise Backup delivers:
NEW! Continuous monitoring – Monitor the progress and disk space usage
“Hot” Online Backups – Backups take place entirely online, without interrupting MySQL transactions
High Performance – Save time with faster backup and recovery
Incremental Backup – Backup only data that has changed since the last backup
Partial Backup – Target particular tables or tablespaces
Compression – Cut costs by reducing storage requirements up to 90%
Backup to Tape – Stream backup to tape or other media management solutions
Fast Recovery – Get servers back online and create replicated servers
Point-in-Time Recovery (PITR) – Recover to a specific transaction
Partial restore – Recover targeted tables or tablespaces
Restore to a separate location – Rapidly create clones for fast replication setup
Reduce Failures – Use a proven high quality solution from the developers of MySQL
Multi-platform – Backup and Restore on Linux, Windows, Mac & Solaris
(from: http://ift.tt/Q5GptV)
While mysqldump is free to use, MEB is part of MySQL’s Enterprise Edition (EE) – so you need a license to use it. But if you are using MySQL in a production environment, you might want to look at EE, as:
MySQL Enterprise Edition includes the most comprehensive set of advanced features, management tools and technical support to achieve the highest levels of MySQL scalability, security, reliability, and uptime. It reduces the risk, cost, and complexity in developing, deploying, and managing business-critical MySQL applications.
(from: http://ift.tt/Q5GrC6)
Before using MEB and backing up your database for the first time, you will need some information:
Information to gather – Where to Find It – How It Is Used
Path to MySQL configuration file – Default system locations, hardcoded application default locations, or from –defaults-file option in mysqld startup script. – This is the preferred way to convey database configuration information to the mysqlbackup command, using the –defaults-file option. When connection and data layout information is available from the configuration file, you can skip most of the other choices listed below.
MySQL port – MySQL configuration file or mysqld startup script. Used to connect to the database instance during backup operations. Specified via the –port option of mysqlbackup. –port is not needed if available from MySQL configuration file. Not needed when doing an offline (cold) backup, which works directly on the files using OS-level file permissions.
Path to MySQL data directory – MySQL configuration file or mysqld startup script. – Used to retrieve files from the database instance during backup operations, and to copy files back to the database instance during restore operations. Automatically retrieved from database connection for hot and warm backups. Taken from MySQL configuration file for cold backups.
ID and password of privileged MySQL user – You record this during installation of your own databases, or get it from the DBA when backing up databases you do not own. Not needed when doing an offline (cold) backup, which works directly on the files using OS-level file permissions. For cold backups, you log in as an administrative user. – Specified via the –password option of the mysqlbackup. Prompted from the terminal if the –password option is present without the password argument.
Path under which to store backup data – You choose this. See Section 3.1.3, “Designate a Location for Backup Data” for details. – By default, this directory must be empty for mysqlbackup to write data into it, to avoid overwriting old backups or mixing up data from different backups. Use the –with-timestamp option to automatically create a subdirectory with a unique name, when storing multiple sets of backup data under the same main directory.
Owner and permission information for backed-up files (for Linux, Unix, and OS X systems) – In the MySQL data directory. – If you do the backup using a different OS user ID or a different umask setting than applies to the original files, you might need to run commands such as chown and chmod on the backup data. See Section A.1, “Limitations of mysqlbackup Command” for details.
Size of InnoDB redo log files – Calculated from the values of the innodb_log_file_size and innodb_log_files_in_group configuration variables. Use the technique explained for the –incremental-with-redo-log-only option. – Only needed if you perform incremental backups using the –incremental-with-redo-log-only option rather than the –incremental option. The size of the InnoDB redo log and the rate of generation for redo data dictate how often you must perform incremental backups.
Rate at which redo data is generated – Calculated from the values of the InnoDB logical sequence number at different points in time. Use the technique explained for the –incremental-with-redo-log-only option. – Only needed if you perform incremental backups using the –incremental-with-redo-log-only option rather than the –incremental option. The size of the InnoDB redo log and the rate of generation for redo data dictate how often you must perform incremental backups.
(from: http://ift.tt/1eiULqM
For most backup operations, the mysqlbackup command connects to the MySQL server through –user and –password options. If you aren’t going to use the root user, then you will need to create a separate user. Follow these instructions for setting the proper permissions.
All backup-related operations either create new files or reference existing files underneath a specified directory that holds backup data. Choose this directory in advance, on a file system with sufficient storage. (It could even be remotely mounted from a different server.) You specify the path to this directory with the –backup-dir option for many invocations of the mysqlbackup command.
Once you establish a regular backup schedule with automated jobs, it is preferable to keep each backup within a timestamped subdirectory underneath the main backup directory. To make the mysqlbackup command create these subdirectories automatically, specify the –with-timestamp option each time you run mysqlbackup.
For one-time backup operations, for example when cloning a database to set up a replication slave, you might specify a new directory each time, or specify the –force option of mysqlbackup to overwrite older backup files.
(from http://ift.tt/1eiULqF
If you haven’t downloaded and installed mysqlbackup, you may download it from edelivery.oracle.com (registration is required). Install the MySQL Enterprise Backup product on each database server whose contents you intend to back up. You perform all backup and restore operations locally, by running the mysqlbackup command on the same server as the MySQL instance. Information on installation may be found here.
Now that we have gathered all of the required information and installed mysqlbackup, let’s run a simple and easy backup of the entire database. I installed MEB in my /usr/local directory, so I am including the full path of mysqlbackup. I am using the backup-and-apply-log option, which combines the –backup and the –apply-log options into one. The –backup option performs the initial phase of a backup. The second phase is performed later by running mysqlbackup again with the –apply-log option, which brings the InnoDB tables in the backup up-to-date, including any changes made to the data while the backup was running.
$ /usr/local/meb/bin/mysqlbackup –user=root –password –backup-dir=/Users/tonydarnell/hotbackups backup-and-apply-log
MySQL Enterprise Backup version 3.8.2 [2013/06/18] Copyright (c) 2003, 2012, Oracle and/or its affiliates. All Rights Reserved.
mysqlbackup: INFO: Starting with following command line …
/usr/local/meb/bin/mysqlbackup –user=root –password –backup-dir=/Users/tonydarnell/hotbackups backup-and-apply-log Enter password: mysqlbackup: INFO: MySQL server version is ‘5.6.9-rc-log’.
mysqlbackup: INFO: Got some server configuration information from running server.
IMPORTANT: Please check that mysqlbackup run completes successfully.
At the end of a successful ‘backup-and-apply-log’ run mysqlbackup
prints "mysqlbackup completed OK!".
——————————————————————–
Server Repository Options:
——————————————————————–
datadir = /usr/local/mysql/data/
innodb_data_home_dir = /usr/local/mysql/data
innodb_data_file_path = ibdata1:40M:autoextend
innodb_log_group_home_dir = /usr/local/mysql/data
innodb_log_files_in_group = 2
innodb_log_file_size = 5242880
innodb_page_size = 16384
innodb_checksum_algorithm = innodb
innodb_undo_directory = /usr/local/mysql/data/
innodb_undo_tablespaces = 0
innodb_undo_logs = 128
——————————————————————–
Backup Config Options:
——————————————————————–
datadir = /Users/tonydarnell/hotbackups/datadir
innodb_data_home_dir = /Users/tonydarnell/hotbackups/datadir
innodb_data_file_path = ibdata1:40M:autoextend
innodb_log_group_home_dir = /Users/tonydarnell/hotbackups/datadir
innodb_log_files_in_group = 2
innodb_log_file_size = 5242880
innodb_page_size = 16384
innodb_checksum_algorithm = innodb
innodb_undo_directory = /Users/tonydarnell/hotbackups/datadir
innodb_undo_tablespaces = 0
innodb_undo_logs = 128
mysqlbackup: INFO: Unique generated backup id for this is 13742482113579320
mysqlbackup: INFO: Creating 14 buffers each of size 16777216.
130719 11:36:53 mysqlbackup: INFO: Full Backup operation starts with following threads
1 read-threads 6 process-threads 1 write-threads
130719 11:36:53 mysqlbackup: INFO: System tablespace file format is Antelope.
130719 11:36:53 mysqlbackup: INFO: Starting to copy all innodb files…
130719 11:36:53 mysqlbackup: INFO: Copying /usr/local/mysql/data/ibdata1 (Antelope file format).
130719 11:36:53 mysqlbackup: INFO: Found checkpoint at lsn 135380756.
130719 11:36:53 mysqlbackup: INFO: Starting log scan from lsn 135380480.
130719 11:36:53 mysqlbackup: INFO: Copying log…
130719 11:36:54 mysqlbackup: INFO: Log copied, lsn 135380756.
<font color="blue"><i>(I have truncated some of the database and table output to save space)</font></i>
…..
130719 11:36:56 mysqlbackup: INFO: Copying /usr/local/mysql/data/mysql/innodb_index_stats.ibd (Antelope file format).
130719 11:36:56 mysqlbackup: INFO: Copying /usr/local/mysql/data/mysql/innodb_table_stats.ibd (Antelope file format).
130719 11:36:56 mysqlbackup: INFO: Copying /usr/local/mysql/data/mysql/slave_master_info.ibd (Antelope file format).
130719 11:36:56 mysqlbackup: INFO: Copying /usr/local/mysql/data/mysql/slave_relay_log_info.ibd (Antelope file format).
130719 11:36:56 mysqlbackup: INFO: Copying /usr/local/mysql/data/mysql/slave_worker_info.ibd (Antelope file format).
…..
130719 11:36:56 mysqlbackup: INFO: Copying /usr/local/mysql/data/testcert/t1.ibd (Antelope file format).
130719 11:36:56 mysqlbackup: INFO: Copying /usr/local/mysql/data/testcert/t3.ibd (Antelope file format).
…..
130719 11:36:57 mysqlbackup: INFO: Copying /usr/local/mysql/data/watchdb/watches.ibd (Antelope file format).
…..
130719 11:36:57 mysqlbackup: INFO: Completing the copy of innodb files.
130719 11:36:58 mysqlbackup: INFO: Preparing to lock tables: Connected to mysqld server.
130719 11:36:58 mysqlbackup: INFO: Starting to lock all the tables…
130719 11:36:58 mysqlbackup: INFO: All tables are locked and flushed to disk
130719 11:36:58 mysqlbackup: INFO: Opening backup source directory ‘/usr/local/mysql/data/’
130719 11:36:58 mysqlbackup: INFO: Starting to backup all non-innodb files in subdirectories of ‘/usr/local/mysql/data/’
…..
130719 11:36:58 mysqlbackup: INFO: Copying the database directory ‘comicbookdb’
…..
130719 11:36:59 mysqlbackup: INFO: Copying the database directory ‘mysql’
130719 11:36:59 mysqlbackup: INFO: Copying the database directory ‘performance_schema’
…..
130719 11:36:59 mysqlbackup: INFO: Copying the database directory ‘test’
…..
130719 11:36:59 mysqlbackup: INFO: Copying the database directory ‘watchdb’
130719 11:36:59 mysqlbackup: INFO: Completing the copy of all non-innodb files.
130719 11:37:00 mysqlbackup: INFO: A copied database page was modified at 135380756.
(This is the highest lsn found on page)
Scanned log up to lsn 135384397.
Was able to parse the log up to lsn 135384397.
Maximum page number for a log record 375
130719 11:37:00 mysqlbackup: INFO: All tables unlocked
130719 11:37:00 mysqlbackup: INFO: All MySQL tables were locked for 1.589 seconds.
130719 11:37:00 mysqlbackup: INFO: Full Backup operation completed successfully.
130719 11:37:00 mysqlbackup: INFO: Backup created in directory ‘/Users/tonydarnell/hotbackups’
130719 11:37:00 mysqlbackup: INFO: MySQL binlog position: filename mysql-bin.000013, position 85573
————————————————————-
Parameters Summary ————————————————————-
Start LSN : 135380480
End LSN : 135384397
————————————————————-
mysqlbackup: INFO: Creating 14 buffers each of size 65536.
130719 11:37:00 mysqlbackup: INFO: Apply-log operation starts with following threads
1 read-threads 1 process-threads
130719 11:37:00 mysqlbackup: INFO: ibbackup_logfile’s creation parameters:
start lsn 135380480, end lsn 135384397,
start checkpoint 135380756.
mysqlbackup: INFO: InnoDB: Starting an apply batch of log records to the database…
InnoDB: Progress in percent: 0 1 …. 99 Setting log file size to 5242880
Setting log file size to 5242880
130719 11:37:00 mysqlbackup: INFO: We were able to parse ibbackup_logfile up to
lsn 135384397.
mysqlbackup: INFO: Last MySQL binlog file position 0 85573, file name mysql-bin.000013
130719 11:37:00 mysqlbackup: INFO: The first data file is ‘/Users/tonydarnell/hotbackups/datadir/ibdata1’
and the new created log files are at ‘/Users/tonydarnell/hotbackups/datadir’
130719 11:37:01 mysqlbackup: INFO: Apply-log operation completed successfully.
130719 11:37:01 mysqlbackup: INFO: Full backup prepared for recovery successfully.
mysqlbackup completed OK!
Now, I can take a look at the backup file that was created:
root@macserver01: $ pwd
/Users/tonydarnell/hotbackups
root@macserver01: $ ls -l
total 8
-rw-r–r– 1 root staff 351 Jul 19 11:36 backup-my.cnf
drwx—— 21 root staff 714 Jul 19 11:37 datadir
drwx—— 6 root staff 204 Jul 19 11:37 meta
$ ls -l datadir
total 102416
drwx—— 5 root staff 170 Jul 19 11:36 comicbookdb
-rw-r—– 1 root staff 5242880 Jul 19 11:37 ib_logfile0
-rw-r—– 1 root staff 5242880 Jul 19 11:37 ib_logfile1
-rw-r–r– 1 root staff 4608 Jul 19 11:37 ibbackup_logfile
-rw-r–r– 1 root staff 41943040 Jul 19 11:37 ibdata1
drwx—— 88 root staff 2992 Jul 19 11:36 mysql
drwx—— 55 root staff 1870 Jul 19 11:36 performance_schema
drwx—— 3 root staff 102 Jul 19 11:36 test
drwx—— 30 root staff 1020 Jul 19 11:36 testcert
drwx—— 19 root staff 646 Jul 19 11:36 watchdb
root@macserver01: $ ls -l meta
total 216
-rw-r–r– 1 root staff 90786 Jul 19 11:37 backup_content.xml
-rw-r–r– 1 root staff 5746 Jul 19 11:36 backup_create.xml
-rw-r–r– 1 root staff 265 Jul 19 11:37 backup_gtid_executed.sql
-rw-r–r– 1 root staff 321 Jul 19 11:37 backup_variables.txt
As you can see, the backup was created in /Users/tonydarnell/hotbackups. If I wanted to have a unique folder for this backup, I can use the –with-timestamp.
The –with-timestamp option places the backup in a subdirectory created under the directory you specified above. The name of the backup subdirectory is formed from the date and the clock time of the backup run.
(from: http://ift.tt/1eiULH9)
I will run the same backup command again, but with the –with-timestamp option:
(I am not going to duplicate the entire output – but I will only show you the output where it creates the sub-directory under /Users/tonydarnell/hotbackups)$ /usr/local/meb/bin/mysqlbackup –user=root –password –backup-dir=/Users/tonydarnell/hotbackups backup-and-apply-log –with-timestamp
……
130719 11:49:54 mysqlbackup: INFO: The first data file is ‘/Users/tonydarnell/hotbackups/2013-07-19_11-49-48/datadir/ibdata1’
<font color="blue">and the new created log files are at ‘/Users/tonydarnell/hotbackups/2013-07-19_11-49-48/datadir'</font>
130719 11:49:54 mysqlbackup: INFO: Apply-log operation completed successfully.
130719 11:49:54 mysqlbackup: INFO: Full backup prepared for recovery successfully.
mysqlbackup completed OK!
So, I ran the backup again to get a unique directory. Instead of the backup files/directories being placed in /Users/tonydarnell/hotbackups, it created a sub-directory with a timestamp for the directory name:
$ pwd
/Users/tonydarnell/hotbackups
root@macserver01: $ ls -l
total 0
drwx—— 5 root staff 170 Jul 19 11:49 2013-07-19_11-49-48
$ ls -l 2013-07-19_11-49-48
total 8
-rw-r–r– 1 root staff 371 Jul 19 11:49 backup-my.cnf
drwx—— 21 root staff 714 Jul 19 11:49 datadir
drwx—— 6 root staff 204 Jul 19 11:49 meta
Note: If you don’t use the –backup-and-apply-log option you will need to read this: Immediately after the backup job completes, the backup files might not be in a consistent state, because data could be inserted, updated, or deleted while the backup is running. These initial backup files are known as the raw backup.
You must update the backup files so that they reflect the state of the database corresponding to a specific InnoDB log sequence number. (The same kind of operation as crash recovery.) When this step is complete, these final files are known as the prepared backup.
During the backup, mysqlbackup copies the accumulated InnoDB log to a file called ibbackup_logfile. This log file is used to “roll forward” the backed-up data files, so that every page in the data files corresponds to the same log sequence number of the InnoDB log. This phase also creates new ib_logfiles that correspond to the data files.
The mysqlbackup option for turning a raw backup into a prepared backup is –apply-log. You can run this step on the same database server where you did the backup, or transfer the raw backup files to a different system first, to limit the CPU and storage overhead on the database server.
Note: Since the –apply-log operation does not modify any of the original files in the backup, nothing is lost if the operation fails for some reason (for example, insufficient disk space). After fixing the problem, you can safely retry –apply-log and by specifying the –force option, which allows the data and log files created by the failed –apply-log operation to be overwritten.
For simple backups (without compression or incremental backup), you can combine the initial backup and the –apply-log step using the option –backup-and-apply-log.
(from: http://ift.tt/1BeG5Oj)
One file that was not copied was the my.cnf file. You will want to have a separate script to copy this at regular intervals. If you put the mysqlbackup command in a cron or Windows Task Manager job, you can add a way to copy the my.cnf file as well.
Now that we have a completed backup, we are going to copy the backup files and the my.cnf file over to a different server to restore the databases. We will be using a server that was setup as a slave server to the server where the backup occurred. If you need to restore the backup to the same server, you will need to refer to this section of the mysqlbackup manual. I copied the backup files as well as the my.cnf file to the new server:
# pwd
/Users/tonydarnell/hotbackups
# ls -l
total 16
drwxrwxrwx 5 tonydarnell staff 170 Jul 19 15:38 2013-07-19_11-49-48
On the new server (where I will restore the data), I shutdown the mysqld process (mysqladmin -uroot -p shutdown), copied the my.cnf file to the proper directory, and now I can restore the database to the new server, using the copy-back option. The copy-back option requires the database server to be already shut down, then copies the data files, logs, and other backed-up files from the backup directory back to their original locations, and performs any required postprocessing on them.
(from: http://ift.tt/1eiULHh)
# /usr/local/meb/bin/mysqlbackup –defaults-file=/etc/my.cnf –backup-dir=/Users/tonydarnell/hotbackups/2013-07-19_11-49-48 copy-back
MySQL Enterprise Backup version 3.8.2 [2013/06/18] Copyright (c) 2003, 2012, Oracle and/or its affiliates. All Rights Reserved.
mysqlbackup: INFO: Starting with following command line …
/usr/local/meb/bin/mysqlbackup –defaults-file=/etc/my.cnf –backup-dir=/Users/tonydarnell/hotbackups/2013-07-19_11-49-48 copy-back IMPORTANT: Please check that mysqlbackup run completes successfully.
At the end of a successful ‘copy-back’ run mysqlbackup
prints "mysqlbackup completed OK!".
——————————————————————–
Server Repository Options:
——————————————————————–
datadir = /usr/local/mysql/data
innodb_data_home_dir = /usr/local/mysql/data
innodb_data_file_path = ibdata1:40M:autoextend
innodb_log_group_home_dir = /usr/local/mysql/data
innodb_log_files_in_group = 2
innodb_log_file_size = 5M
innodb_page_size = Null
innodb_checksum_algorithm = innodb
——————————————————————–
Backup Config Options:
——————————————————————–
datadir = /Users/tonydarnell/hotbackups/2013-07-19_11-49-48/datadir
innodb_data_home_dir = /Users/tonydarnell/hotbackups/2013-07-19_11-49-48/datadir
innodb_data_file_path = ibdata1:40M:autoextend
innodb_log_group_home_dir = /Users/tonydarnell/hotbackups/2013-07-19_11-49-48/datadir
innodb_log_files_in_group = 2
innodb_log_file_size = 5242880
innodb_page_size = 16384
innodb_checksum_algorithm = innodb
innodb_undo_directory = /Users/tonydarnell/hotbackups/2013-07-19_11-49-48/datadir
innodb_undo_tablespaces = 0
innodb_undo_logs = 128
mysqlbackup: INFO: Creating 14 buffers each of size 16777216.
130719 15:54:41 mysqlbackup: INFO: Copy-back operation starts with following threads
1 read-threads 1 write-threads
130719 15:54:41 mysqlbackup: INFO: Copying /Users/tonydarnell/hotbackups/2013-07-19_11-49-48/datadir/ibdata1.
…..
130719 15:54:42 mysqlbackup: INFO: Copying /Users/tonydarnell/hotbackups/2013-07-19_11-49-48/datadir/comicbookdb/comics.ibd.
…..
130719 15:54:42 mysqlbackup: INFO: Copying /Users/tonydarnell/hotbackups/2013-07-19_11-49-48/datadir/mysql/innodb_index_stats.ibd.
130719 15:54:42 mysqlbackup: INFO: Copying /Users/tonydarnell/hotbackups/2013-07-19_11-49-48/datadir/mysql/innodb_table_stats.ibd.
130719 15:54:42 mysqlbackup: INFO: Copying /Users/tonydarnell/hotbackups/2013-07-19_11-49-48/datadir/mysql/slave_master_info.ibd.
130719 15:54:42 mysqlbackup: INFO: Copying /Users/tonydarnell/hotbackups/2013-07-19_11-49-48/datadir/mysql/slave_relay_log_info.ibd.
130719 15:54:42 mysqlbackup: INFO: Copying /Users/tonydarnell/hotbackups/2013-07-19_11-49-48/datadir/mysql/slave_worker_info.ibd.
…..
130719 15:54:43 mysqlbackup: INFO: Copying /Users/tonydarnell/hotbackups/2013-07-19_11-49-48/datadir/watchdb/watches.ibd.
…..
130719 15:54:43 mysqlbackup: INFO: Copying the database directory ‘comicbookdb’
…..
130719 15:54:43 mysqlbackup: INFO: Copying the database directory ‘mysql’
130719 15:54:43 mysqlbackup: INFO: Copying the database directory ‘performance_schema’
…..
130719 15:54:43 mysqlbackup: INFO: Copying the database directory ‘test’
…..
130719 15:54:43 mysqlbackup: INFO: Copying the database directory ‘watchdb’
130719 15:54:43 mysqlbackup: INFO: Completing the copy of all non-innodb files.
130719 15:54:43 mysqlbackup: INFO: Copying the log file ‘ib_logfile0’
130719 15:54:43 mysqlbackup: INFO: Copying the log file ‘ib_logfile1’
130719 15:54:44 mysqlbackup: INFO: Copy-back operation completed successfully.
130719 15:54:44 mysqlbackup: INFO: Finished copying backup files to ‘/usr/local/mysql/data’
mysqlbackup completed OK!
I can now restart MySQL. I have a very small database (less than 50 megabytes). But it took less than a minute to restore the database. If I had to rebuild my database using mysqldump, it would take a lot longer. If you have a very large database, the different in using mysqlbackup and mysqldump could be in hours. For example, a 32-gig database with 33 tables takes about eight minutes to restore with mysqlbackup. Restoring the same database with a mysqldump file takes over two hours.
An easy way to check to see if the databases match (assuming that I haven’t added any new records in any of the original databases – which I haven’t), I can use one of the MySQL Utilities – mysqldbcompare. I wrote about how to do this in an earlier blog about using it to test two replicated databases, but it will work here as well – see Using MySQL Utilities Workbench Script mysqldbcompare To Compare Two Databases In Replication.
The mysqldbcompare utility “compares the objects and data from two databases to find differences. It identifies objects having different definitions in the two databases and presents them in a diff-style format of choice. Differences in the data are shown using a similar diff-style format. Changed or missing rows are shown in a standard format of GRID, CSV, TAB, or VERTICAL.” (from: mysqldbcompare — Compare Two Databases and Identify Differences)
Some of the syntax may have changed for mysqldbcompare since I wrote that blog, so you will need to reference the help notes for mysqldbcompare. You would need to run this for each of your databases.
$ mysqldbcompare –server1=scripts:scripts999@192.168.1.2 –server2=scripts:scripts999@192.168.1.123 –run-all-tests –difftype=context comicbookdb:comicbookdb
# server1 on 192.168.1.2: … connected.
# server2 on 192.168.1.123: … connected.
# Checking databases comicbookdb on server1 and comicbookdb on server2
Defn Row Data Type Object Name Diff Count Check ————————————————————————— TABLE comics pass pass pass Databases are consistent.
# …done
You can try and run this for the mysql database, but you may get a few errors regarding the mysql.backup_history and mysql.backup_progress tables:
$ mysqldbcompare –server1=scripts:scripts999@192.168.1.2 –server2=scripts:scripts999@192.168.1.123 –run-all-tests –difftype=context mysql:mysql
# server1 on 192.168.1.2: … connected.
# server2 on 192.168.1.123: … connected.
# Checking databases mysql on server1 and mysql on server2
Defn Row Data Type Object Name Diff Count Check ————————————————————————— TABLE backup_history pass FAIL SKIP Row counts are not the same among mysql.backup_history and mysql.backup_history.
No primary key found.
TABLE backup_progress pass FAIL SKIP Row counts are not the same among mysql.backup_progress and mysql.backup_progress.
No primary key found.
TABLE columns_priv pass pass pass TABLE db pass pass pass TABLE event pass pass pass TABLE func pass pass pass TABLE general_log pass pass SKIP No primary key found.
TABLE help_category pass pass pass TABLE help_keyword pass pass pass TABLE help_relation pass pass pass TABLE help_topic pass pass pass TABLE innodb_index_stats pass pass pass TABLE innodb_table_stats pass pass pass TABLE inventory pass pass pass TABLE ndb_binlog_index pass pass pass TABLE plugin pass pass pass TABLE proc pass pass pass TABLE procs_priv pass pass pass TABLE proxies_priv pass pass pass TABLE servers pass pass pass TABLE slave_master_info pass pass pass TABLE slave_relay_log_info pass pass pass TABLE slave_worker_info pass pass pass TABLE slow_log pass pass SKIP No primary key found.
TABLE tables_priv pass pass pass TABLE time_zone pass pass pass TABLE time_zone_leap_second pass pass pass TABLE time_zone_name pass pass pass TABLE time_zone_transition pass pass pass TABLE time_zone_transition_type pass pass pass TABLE user pass pass pass Database consistency check failed.
# …done
For example, when you compare the mysql.backup_history tables, the original database will have two entries – as I ran mysqlbackup twice. But the second backup entry doesn’t get entered until after the backup has occurred, and it isn’t reflected in the backup files.
Original Servermysql&gt; select count(*) from mysql.backup_history;
+———-+
| count(*) |
+———-+
| 2 |
+———-+
1 row in set (0.00 sec)
Restored Servermysql&gt; select count(*) from mysql.backup_history;
+———-+
| count(*) |
+———-+
| 1 |
+———-+
1 row in set (0.00 sec)
For the mysql.backup_progress tables, the original database has ten rows, while the restored database has seven.
There are many options for using mysqlbackup, including (but not limited to) incremental backup, partial backup , compression, backup to tape, point-in-time recovery (PITR), partial restore, etc. If you are running MySQL in a production environment, then you should look at MySQL Enterprise Edition, which includes MySQL Enterprise Backup. Of course, you should always have a backup and recovery plan in place. Finally, if and when possible, practice restoring your backup on a regular basis, to make sure that if your server crashes, you can restore your database quickly.
 
Tony Darnell is a Principal Sales Consultant for MySQL, a division of Oracle, Inc. MySQL is the world’s most popular open-source database program. Tony may be reached at info [at] ScriptingMySQL.com and on LinkedIn.
via Planet MySQL
Creating and Restoring Database Backups With mysqldump and MySQL Enterprise Backup – Part 2 of 2