Meta Beta Geek

For those of you who might have missed my previous entry and the history behind passwordmeter.com, feel free to read about it here. For the rest of you, you’ll be happy to know that I finally got around to adjusting the algorithms in the Password Meter script. My original update was to be a single html page with javascript, css and even images embedded. This would have made it really easy to download and deploy in virtually any environment. However, due to a lack of support for base64 image conversion in earlier versions of Internet Explorer, I was forced to include a separate directory for images. Since I had to create a separate directory for images, I opted to extract the Javascript and CSS code as well, and place each into its own respective directory. So the final download is actually a zip file that contains the main page and all supporting scripts, images and stylesheets, plus a copy of the GPL license.

In addition to generally cleaning up the code a bit and adding penalties for repeat symbols, the main update to the script was done to alter the method used to calculate deductions for repeat characters. This has been an issue since version 1.03 was deployed and was never addressed until now. I originally used an exponential formula to penalize users for adding the same character more than once to their password. As the number of identical characters increased, the penalty became more and more severe – to the point where users would end up with a score of zero, despite having a reasonably difficult password. I realized the flaw shortly after I deployed it but never got around to fixing it due to other projects getting in the way.

In the latest version, repeat characters are still penalized. But the formula is now based on proximity to other identical characters where further distance means less deduction. I also accounted for the total number of unique characters and weighed that number into the calculation as well. So if you have a 12 character password that consists entirely of unique alpha-numerics and symbols, then add a string of 14 “x”s to the end, the deduction penalty for the repeat characters is significantly reduced. Being that the password meter utility is run entirely on the client side, I’m still limited to the tools that Javascript can provide, so it’s still not quite as accurate as I’d like it to be. But this new version should be a lot more accurate than the previous releases in terms of applied penalties and total score calculation.

I am planning to rebuild the passwordmeter.com site some time in the next week or so. Nothing major is planned but I need to revamp it a bit to support the new code base as well as the ability to add new blog entries that are specific to the password meter code. The download link will also be changed so that it points back to the software repository here at Meta Beta Geek. In the mean time, while you’re waiting for the passwordmeter.com site to be updated, you can either play with the demo site, or download the code directly. For downloading, just select the “MBG PWDMeter Package” from the software list. Please let me know if you notice any glaring bugs or issues with the new 2.0 release.

UPDATE:

The scripts and links at passwordmeter.com have now been updated.

Cheers,

Jeff @ Meta Beta Geek

With a number of different data-driven projects on my plate both at home and at work, I find myself ever more impressed with the built-in abilities of MySQL.  Like many open source software products, it’s available at no cost as long as it’s used under the guidelines of the provided OS license.  But MySQL has proven again and again that it’s no ordinary piece of OS software.  Where other database software companies are charging hundreds or thousands of dollars for licensing fees, MySQL continues to offer enterprise-level features and performance without the typically-associated costs.

I’m a bit of a newcomer to MySQL, I’ll admit.  I only jumped aboard the PHP-MySQL ship some three years back.  Since that time, the speed and depth of my learning has varied from day to day.  Much of what I learn is based on the particular requirements of the project at hand.  The more demanding and intricate the project, the more digging I’m forced to do.  Not to sound to “self-help-ish”, but the end result of this process is typically a better, more knowledgeable me.  Some of the oddest requests in the past have actually turned out to be some of the coolest features in an app.  And I’m learning more in the process so it’s really a win-win.

In a never-ending effort to improve the “product”, I am always seeking new knowledge.  From a purely logical standpoint, the more I know, the better the application should, in turn, operate and perform.  As an example, recent changes to my Scaffolder utility resulted in a highly-improved product and some cool new tools for my coding arsenal.  A coworker of mine recently pointed out the possibility of performing multiple record inserts using a single SQL statement.  Although the concept seemed like a good one, I’d always assumed this was not possible.  As it turns out, MySQL has built-in support for this very feature.  Some brief testing resulted in a startling revelation: the performance difference between inserting 1000 records as separate queries versus a single query string was absolutely amazing!

For those of you who are unfamiliar with this built-in capability, I highly recommend reading up on it and then immediately converting any of your bulk insert queries to use this new format.  The basic gist from the MySQL site is as follows:

INSERT statements that use VALUES syntax can insert multiple rows. To do this, include multiple lists of column values, each enclosed within parentheses and separated by commas. The values list for each row must be enclosed within parentheses. Example:

INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(4,5,6),(7,8,9)

Another little tip I recently picked up was the FIND_IN_SET option.  This built-in function allows you to search and match a comma-separated string of values (saved within a database field) for one particular value.  Up ’til now, I had relied on the following alternative, but much more complicated, syntax:

field REGEXP CONCAT('(^|,)(',REPLACE(value,',','|'),')($|,)')

FIND_IN_SET accomplishes the exact same thing without the overly-bloated syntax and unnecessary conversions/operations. The above statement can easily be converted to use FIND_IN_SET in the following manner:

FIND_IN_SET(value,field)

I don’t have the tools required to determine whether there is a significant difference in performance between these two methods. But based on what I’ve heard about MySQL’s REGEXP capabilities, combined with the reduction in syntax and operations, I’m assuming the latter option will result in a speedier delivery.  I did test the difference between multiple individual insert statements and a single bulk insert statement and the performance difference was night and day.  Where a large amount of inserts are required, the single, bulk insert statement will reduce performance overhead tremendously, saving developers and end users a lot of time.

There’s not a whole lot more detail on either of these two options but if you’re interested in viewing the documentation direct from the source you can read about these options using the following links:

MySQL’s BULK INSERT
MySQL’s FIND_IN_SET

Since learning of these new options, the Scaffolder project has been updated accordingly, incorporating both of these options into its source code. The performance of the Bulk Insert option within Scaffolder is now significantly improved. Testing showed that 10,000 random and data-type specific records could be inserted in roughly 40 seconds flat. And those tests were run on a development desktop platform. Server performance would undoubtedly be even better.

I guess my lesson here is that there’s never a point where we stop learning. My projects provide me with a guide and driver that takes me to new places each and every day. And I sincerely hope the newness never goes away. I love a challenge and constantly learning something new. I’ve given up on the idea of “catching up” with the existing development community. I realize now that this process is constantly evolving and will never really reach a stopping point. I guess that’s what makes it so much fun.

This is just a quick note for all the MBG Scaffolder users.  Version 2.0.6 has just been released with a variety of bug fixes and updates.  But the most prominent feature now available is the option of generating image thumbnails during file uploads.  I had planned to add this option in at some distant point in the future.  But the upgrade happened sooner than later because I required this feature for another project I’m working on at the moment.  So as users, you can just sit back and reap the rewards!

I still have to update the help documentation and the feature list but using the thumbnail generator is pretty simple.  There’s a new option available under the Files tab now with a list of image sizes from which to select.  Currently, you can choose up to 12 different options ranging from 75×75 pixels to 1600×1200 pixels.  I guess the larger images are technically not thumbnails but you get the idea.  If nothing is selected from this list, no thumbnails are created.  Removing an uploaded file from an existing record will automatically remove any related thumbnails too, making it super simple to use this feature.

If you still don’t know what MBG Scaffolder is all about, check out the introductory post for all the details.  If you just want the latest version available, you can download it here.  Again thanks for testing and all the constructive feedback.  Every bit helps me to improve the end product.

Cheers,

Jeff @ Meta Beta Geek

Having worked within a variety of different programming languages, I realize that there are obvious syntactic differences from one language to the next.  By the variance is not just with syntax.  Standards for variable and function declaration differ quite a bit as well.  Even within a single language, opinions vary as to which standards are considered best practice.  Since I have every intention of releasing my code to the open source community, I’d like to develop a standard coding format that conforms as much to the normal standards as possible.

My second official release is a PHP-MySQL scaffolding utility known as MBG Scaffolder.  The downloadable version of this software consists of a single file where all comments and excess white space have been removed.  This assists end-users because the download size is smaller and the utility takes less time to load when in use.  The obvious disadvantage to this format is that it’s not entirely user-friendly and certainly doesn’t lend itself to future development by outside programmers.  So the next step in this process is to release a non-minified version of the code with white space and full comments intact.

The current MBG Scaffolder package does include end-user help documentation.  These docs are designed to assist users with learning the ins and outs of the web interface only.  End users are typically not concerned with how back-end code works, only that it does so without errors.  But the current release of this package is decidedly lacking in useable developer documentation.  I hope to remedy this in future releases to make it easier for other developers to utilize this package and even recommend changes to help grow and improve the code base.

With that in mind, I’m interested to know if there’s a single resource available (hard copy or online), that outlines universal coding standards for PHP developers.  I’m putting this out to the rest of the development community in hopes of getting some useful feedback.  My specific focus at the moment is on comments within my code since I’m trying to assist developers as much as possible.  From my brief research, it seems that many developers follow the JavaDocs example for commenting.  But is the norm for the majority of developers?  If not, what are some other recommended styles for commenting within PHP?

UPDATE:

In my brief research I came across several good resources that may assist other PHP developers out there.  On the PHP.net site, under the Pear Manual, there is a great document that outlines sample comments that they recommend for all PHP code.  As I noted previously, these standards are based on JavaDocs standards with only a few exceptions.  I discovered another well-versed set of standards written by Fredrik Kristiansen of DB MediaLab in Norway.  His version appears to be a PHP-based translation of standards based on Todd Hoff’s C++ coding standards.  Another possible resource for developers is the standards written for working within the Zend PHP framework.  However, these docs appear to be somewhat Zend-specific and are a bit more stringent in what is and is not considered acceptable.  But as a loose guide it might prove useful.

Sticking with the first resource under the Pear manual, PHP offers a piece of software that will automatically parse your source code and effectively output a working manual based on the comments within the code.  The format for comments must comply with pre-established standards though so this won’t work with just any code.  The software is known as phpDocumentor and is available for free to any developers who wish to use it.  There are probably other good resources for coding practices to be found.  These are just a few of the ones I discovered that seemed to conform to the standards I’ve seen in the past.  Feel free to leave feedback on other good resources if you know of any others.

If you haven’t already heard, I finally got around to tweaking the MBG Scaffolder code enough that I felt it was ready for beta release.  The MBG Scaffolder package has now been officially released under GNU’s general public license.  For those unfamiliar with the MBG Scaffolder project, please check out my previous post for download and demo links, as well as a complete list of features available.

For this particular project, I opted to make the code available under open source licenses because I’m a big believer in open source software and I really feel this project has something to offer.  Hopefully the latter becomes reality and someone else can finally benefit from my efforts, like I have from others all these years.  With the exception of Password Meter, the MBG Scaffolder package is the first major piece of code I’ve released under the GPL.  But I certainly don’t plan for it to be the last.  I have several other PHP classes, as well as a Javascript or two that I hope to make available under GPL as well.

In my opinion, open source projects are a huge benefit to developers because they get the code out there for others to use and review.  This process lends itself quite nicely to project expansion through other developers and community feedback.  Typically, the more popular a project is, the better the code and feature-set because there’s a genuine interest in maintaining and maturing the final product.  I realize that most people these days are looking to make money whenever possible, but I honestly wish more companies would look into releasing their code under OS licensing.  To me, it seems like a win-win situation.

To answer a lingering questions, some of you may be wondering why I’ve built yet another PHP-MySQL scaffolding tool.  I found a few other scaffolding utilities out there that did some of what I needed, but most of these other apps weren’t very intuitive and required a lot of modifications to the back-end database to make them work correctly.  The MBG Scaffolder was designed with minimal requirements in mind.

With few exceptions, the MBG Scaffolder utility is fully configured and ready to go.  Drop it in a web-accessible directory and using the proper credentials, you can easily access any MySQL database server using a fully qualified domain name.  It’s just that easy.  The only real requirements come in the form of PHP and MySQL version support and some minor table comments to support advanced joining options.  The following is a rough gist of MBG Scaffolder’s abilities:

It’s pretty easy to talk up my own product though, so I’m looking for testers and feedback to make the MBG Scaffolder even better.  My testing has been pretty limited so far so I’m looking for more worldly feedback based on database parameters outside of my own controlled environment.  Quite a few updates were included in the last push so there’s still a possibility of some bugs.  But I did my best to eliminate any issues I came across.

Please feel free download the package and give it a try and let me know what you think.  If you don’t have access to a PHP-enabled web server, check out the demo site to get a feel for MBG Scaffolder’s abilities.  The only feature that’s missing on the demo is the file uploading.  I’ve blocked this for security reasons.  But file uploads are fully enabled in the download-able release.  Don’t forget to pass this on to anyone you know that may have need for a PHP-MySQL scaffolding utility.

Thanks in advance and have fun!

Jeff @ Meta Beta Geek

The latest revision of MBG Scaffolder (v2.0.5) has now been pushed to production and for the moment I have no plans for any additional major updates.  So while I’m sorting out licensing issues and preparing the MBG Scaffolder code for public release, I’m reviewing some of my other projects to determine what’s next on my list of priorities.  There are a number of PHP and Javascript pieces in particular that come to mind, but most of these are fairly recent and don’t require a huge amount of changes prior to release.  So I started digging through my code archives to see if any older pieces were due for a review.

One of my earlier works, a piece of code written over two years ago, still manages to generate quite a lot of Internet traffic.  Password Meter is a Javascript project that I literally sat down and wrote in a single weekend.  At that time, it was designed as nothing more than a fun piece of code for users to play with and explore.  I never imagined that it would become as popular a resource as it has, achieving over a thousand hits per day at its current location, www.passwordmeter.com.  The program was designed to rate a user’s password and provide instantaneous visual feedback related to the password’s strength in terms of crackability.

Since the Password Meter code was initially released under general public license (GPL), I have received quite a lot of feedback related to the internal algorithms used to calculate password strength.  I’ve been aware for quite some time now that this portion of the code is in need of a revamp.  I simply haven’t had the time or energy to deal with it.  So the code has remained largely unchanged since its initial release, an issue I hope to remedy in the near future.

In terms of project scope, the Password Meter script itself is not terribly long, so I don’t expect the re-write to take an overly long time.  But I also tend to get rather entrenched in projects like this, so the end result may be quite a bit more complicated than the first release.  For the sake of keeping things simple though, my current plan is to fix the major issues and re-release the code as a new revision. For the time being, even though Meta Beta Geek was intended to track all my coding projects, the source code for Password Meter will remain at its current location.  Any future revisions or modifications to the code will most likely be referenced and tracked here at Meta Beta Geek.

That’s it for updates at the moment.  Stay tuned for a new release of Password Meter very soon, and don’t forget to leave a comment or two if you have suggestions for code or site improvements.  Happy Holidays to you and yours!

UPDATE:

Password Meter version 2.0 is now available for demo or download.

Cheers,

Jeff @ Meta Beta Geek

The MBG Scaffolder package is an open source PHP scaffolding utility designed and modeled (loosely) after the Ruby On Rails scaffolding concept.  Born out of necessity, MBG Scaffolder provided a front-end web interface for quickly manipulating data within MySQL databases.  The first iteration covered the basics, allowing record creation, retrieval, updating and deletion.  It also allowed for automatic table joins using referential rules within the database.  But it was still pretty limited.

About a month ago, MBG Scaffolder underwent a major revamp, gaining a whole lot of additional features in the process.  Effectively reborn, the second iteration of this package exists entirely in a single, mini-fied file of less than 200 kilobytes.  Using MBG Scaffolder is as easy as placing the file in any web-accessible directory and loading the page from a web browser.  The package is written using PHP and Javascript so the web server must have built-in support for PHP (version 5.x or higher) and the client must have Javascript enabled.  Other than that, there are virtually no other dependencies.

For those who are new to the MBG Scaffolder package, consider the following:

Have you ever wanted to join a field containing comma-separated values (record ids) to the records that correspond to those values in a lookup table?  Known as a CSV field type, the option to join lookup tables with comma-separated values is now available, but only using MBG Scaffolder.  Want the ability to control input based on a pre-established list of values from an Enum field type?   New drop list options for all lookup tables, as well as Enum and Set field types are now available, providing users with a friendly list of pre-validated options.  Need to test database performance on a large scale but lack the necessary data?  The new Bulk Insert feature gives developers the ability to insert anywhere from 5 to 1000 random and data-type specific records in mere seconds.  Tired of filling out the same form field information for dozens of records?  Using the MBG Scaffolder’s Insert Plus One option means you only fill the form out one time for as many records as you need.  Check out the complete list of new features (below) for more details.

In case you missed it, the MBG Scaffolder package also includes built-in help documentation.  I tried to make using the class as intuitive as possible, but I also realize that there may be questions regarding specific features.  If you run into any problems, feel free to leave me feedback using the Comment form, or check for answers within the help docs themselves.  And don’t forget that a good number of the front-end features, such as field labels, buttons, and even some of the more commonly seen errors, offer built-in help links to assist users in getting started.  So if you’re dumbstruck by a particular feature, try looking for a helpful link nearby.

The MBG Scaffolder package is now fully available for download here.  You can also test out a live demo of the most recent version using the credentials that follow.  Tables without a primary or unique key are now fully accessible in a read-only format.  The Blogs and Comments tables are fully modifiable so feel free to add, update and remove records.  The Bulk Insert feature makes it really easy to add records back in for testing.  The remaining tables are locked down since they are the referential tables used to feed the primary tables.  Also, just for the record, I’m not trying to recreate PHPMyAdmin so there are currently no plans for database structure (tables, fields, etc) manipulation at the moment.  This tool is designed strictly with CRUD in mind.

The demo requires the following info:

The server name field supports fully-qualified domain names, so you can conceivably connect up with any DNS-registered MySQL server and play around.  Just be aware that there is no SSL or SSH option yet.  So it works similar to PHPMyAdmin in that respect (credentials are passed unencrypted).  I hope to add an option for SSH some time in the future but this will increase the file footprint dramatically so it’s on hold for now.  Also, the regular expression matching for SQL terms in both queries and records is still in need of tweaking but I’m getting there.  Most everything else should be relatively bug-free.  Please let me know if you come across any issues or have any suggestions for features, etc.

I am a coder… a programmer… a developer… a designer… a database administrator… a software engineer.  There are many terms used to describe what it is I do these days.  But I guess the easiest way to describe it is to tell you that I build dynamic web applications.  I’ve actually been working with code in one form or another since the mid-90s but didn’t really get into server-side and data-driven applications until early 2002.  Since then I’ve spent a great deal of time digging around and learning everything I possibly can about coding, development, design, databases and more.  I have an avid interest in anything related to databases or web application programming so any time I get my hands on some new library or coding technique I turn into a two-year-old with a new toy.  And I gotta say… It’s awesome!

Although I plan to stand up my own site at some point, I’m drastically short on time and overloaded with projects at the moment.  So for now, Wordpress has rescued me from the depths of my own digging.  As of this writing, I have several development projects that are fully releasable and several more that are near completion.  I’ve sunk countless hours into the code behind these projects, many of which are now on their 2nd or 3rd full iteration.  But so far, I’m the only one who has benefited from their usefulness.

Having reaped the rewards of open source licensing for many years now, I think it’s high time I return the favor by contributing something useful back to the pool.  The majority of my projects have universal applications, making them great candidates for an open source license.  So hopefully soon, one or more persons out there will be enjoying the fruits of my labor in the form of GPL-released code.  As for this blog… it will serve as the initial location for documentation behind each project with more detailed documentation to follow at a later date.  So check back soon for more updates!