Showing posts with label scripts. Show all posts
Showing posts with label scripts. Show all posts
Yahoo's MySQL Partition Manager is Open Source
Ozzie
The guys at Yahoo released their partition management script on github:
At Yahoo, we manage a massive number of MySQL databases spread across multiple data centers.You can check out the original anoucement at the MySQL@Yahoo blog and the code at its github repo.
We have thousands of databases and each database has many partitioned tables. In order to efficiently create and maintain partitions we developed a partition manager which automatically manages these for you with minimal pre configuration.
2:58 PM
partitions
,
scripts
,
Tools
Get a summary footprint on a MySQL server instance
Ozzie
Landing on an enterprise with ongoing projects mean that servers are often handed to IT staff without complete knowledge of what's inside.
I've built a simple script, scraping from here and there, to gather a summary of relevant information.
Once you've gained remote access to the MySQL instance, you can execute the queries to identify the following information regarding the target database server:
I've built a simple script, scraping from here and there, to gather a summary of relevant information.
Once you've gained remote access to the MySQL instance, you can execute the queries to identify the following information regarding the target database server:
- The host name, what operating system it runs on, the MySQL version installed, default collation of the instance, the installation directory and the data directory;
- How many user databases it hosts, what they're called and the collation used;
- The size of each database as a whole and broken down by storage engine;
- All the tables and their space used ;
- The creation and last modification on each database.
The script is commented and follows the same order as the above topics:
Running this on my test server, I find the following result for the host, operating system and directory locations:
The following result tells me how many databases there are on the instance:
If relevant, I can choose to see the database names and respective collations:
Then, the script gets the size on each database, in MB:
It breaks that size on the different storage engines:
Still on the size reporting, we get each table's size on records as well as indexes:
Finally, it checks for the creation and update dates of the existing databases to help determine if it's important to have a recent backup of them or not:
This script only gives a summary of the overall instance but has enough details to determine the versions of the software, volume of information and usage of the databases.
Photo credit: tableatny@Flickr
5:22 PM
configuration
,
scripts
Developing a MySQL Workbench plugin
Ozzie
The MySQL Workbench tool is great for development and administration tasks. Also it's available on Windows, Linux and Mac OS X which, according to information from third party sources, is more than you can say for most of the other equivalent tools. And Workbench is free.
Having said that, most of the provided functionalities are intuitive and of daily use for developer and DBA staff alike. Moving beyond this rich out-of-the-box features set, Workbench empowers it's users to extend and develop their own custom features. For this purpose, we use both of the following:
I've developed a small proof of concept to learn how to code a custom plugin. The purpose is to provide a simple refactoring feature for the script on the SQL editor (basically it's a find and replace):
To briefly explain the code, first we import all of the workbench (wb) modules, grt and mforms:
To install this plugin, simply open MySQL Workbench, choose the "Scripting > Install Plugin/Module ..." option, browse to the "refactor_grt.py" file location and open it. Close and re-open the tool. There should be a new option on the "Tools > Utilities" menu called "Refactor Selection". Also, the plugin appear on the plugin manager:
You can read another full example regarding MySQL Workbench on how to define a module and define a plugin in Python here:
http://mysqlworkbench.org/workbench/doc/
Having said that, most of the provided functionalities are intuitive and of daily use for developer and DBA staff alike. Moving beyond this rich out-of-the-box features set, Workbench empowers it's users to extend and develop their own custom features. For this purpose, we use both of the following:
- GRT: Generic RunTime is the internal system used by Workbench to hold model document data. It is also the mechanism by which Workbench can interact with Modules and Plugins.
- MForms: MForms is a small GUI toolkit library written for use in the MySQL Workbench project.
I've developed a small proof of concept to learn how to code a custom plugin. The purpose is to provide a simple refactoring feature for the script on the SQL editor (basically it's a find and replace):
To briefly explain the code, first we import all of the workbench (wb) modules, grt and mforms:
# import the wb moduleThen we declare the module information, its name, author and version:
from wb import *
# import the grt module
import grt
# import the mforms module for GUI stuff
import mforms
# define this Python module as a GRT module
ModuleInfo = DefineModule(name="Refactor", author="mjlmo", version="0.1")
Next we specify plugin internal name, tha caption to use on the workbench menu, what input we'll need to work on and which menu we want the feature to be available in:
@ModuleInfo.plugin("wb.text.refactor", caption = "Refactor Selection", input=[wbinputs.currentQueryBuffer()], pluginMenu= "SQL/Utilities")Finally, we insert the code to perform tha task in hand, in this case replace a selected text on the SQL editor with a given one on an input box. If there is no text selected, a message box will appear on screen stating that. Otherwise, we proceed with the refactoring operation.
@ModuleInfo.export(grt.INT, grt.classes.db_query_QueryBuffer)
To install this plugin, simply open MySQL Workbench, choose the "Scripting > Install Plugin/Module ..." option, browse to the "refactor_grt.py" file location and open it. Close and re-open the tool. There should be a new option on the "Tools > Utilities" menu called "Refactor Selection". Also, the plugin appear on the plugin manager:
You can read another full example regarding MySQL Workbench on how to define a module and define a plugin in Python here:
http://mysqlworkbench.org/workbench/doc/
3:50 PM
development
,
scripts
,
Tools
,
workbench
MySQL sample databases: for testing and training
Ozzie
Sometimes it's useful to have a set of data prepared to be used on a fresh MySQL install for testing purposes. Or you might be preparing some training or workshop and want to prepare examples with fake information. For MySQL there are some choices available:
- Employees Sample Database: provides a combination of a large base of data (approximately 160MB) spread over six separate tables and consisting of 4 million records in total.
- Sakila Sample Database: is intended to provide a standard schema that can be used for examples in books, tutorials, articles, samples, and so forth. Sakila sample database also serves to highlight the latest features of MySQL such as Views, Stored Procedures, and Triggers.
- BIRT Sample Database: The schema is for Classic Models, a retailer of scale models of classic cars. The database contains typical business data such as customers, orders, order line items, products and so on.
In general, you can download a single ZIP file with the schema creation and the data loading in separate scripts.
3:58 PM
development
,
scripts
,
training
Print tab separated values as table using MySQL
Ozzie
Using mysql command line utility to get recordsets, the data rows alignment and line breaks are often a mess. You can use the command line tool on a mysql database server to get a set of rows into an output TSV file like so:
You can put your own filename instead of 'data.csv'. And of course, this script also works for other TSV files that don't come from mysql. To run the script you should install the python tabulate package:
https://pypi.python.org/pypi/tabulate
shell> mysql -u your_user -p < your_statement.sql > data.csvI came up with the following python script to grab the output file and pretty print:You can put your own filename instead of 'data.csv'. And of course, this script also works for other TSV files that don't come from mysql. To run the script you should install the python tabulate package:
https://pypi.python.org/pypi/tabulate
11:29 AM
development
,
scripts
,
Tools
Subscribe to:
Posts
(
Atom
)

