PPRuNe Forums - View Single Post - MySQL SELECT help please...
View Single Post
Old 12th Jun 2017, 01:20
  #2 (permalink)  
G0ULI
 
Join Date: Dec 2013
Location: Norfolk
Age: 68
Posts: 1
Likes: 0
Received 0 Likes on 0 Posts
windriver

This is precisely the reason that large organisations employ quite a few people to check and correct data entry errors and some experts to extract data that is known to be on the system, but cannot be retrieved.

The principle behind database entry methods is consistency. Either the data must be entered in a consistent manner by trained personnel or data is selected from a fixed set of look up tables. A combination of these two methods are used for databases that have to cope with variable data.

You can make searches less specific by searching for just "Rolls" rather than "Rolls Royce" or "Rolls-Royce". Ignoring the capitalisation would also retrieve records that mention, for example, "the aircraft rolls to the right".

Databases have to be sensitive to punctuation and spelling, they are designed to retrieve exactly what is asked for, and only that. That is why it is important to consider consistency of data entry and error checking and detection during data entry when first designing a database.

It is a futile exercise to try and redesign the database after several thousand records have been entered in an ad hoc fashion. each and every record will need to be revisited and edited to conform to whatever standards you decide to apply. The larger the database, the more important it becomes for the integrity of data entry to be maintained.

Clearly your database contains ad hoc information that has been entered in accordance with whatever source it originated from. That is fine. If you need to search for information in that field, variable string searches are the only way of retrieving the records.

It may be of benefit though to ensure that each data record has a consistent set of fields such as date, make, model, registration, and no doubt several others you find of importance. This will at least allow you to retrieve a limited set of records to be examined individually or searched more specifically using a string search when trying to retrieve particular details.

I'm sorry to have to say this, but these design decisions need to be made before you start building a database, not once it is up and running. Then it is too late to make major changes without having to reenter data or migrate the data across to a new system. Both of these solutions involve a lot of work.

Back in the day when Windows was a new innovation on computers, I designed a bespoke database that contained half a dozen free text entry fields along with validated data entry fields that only allowed entries from a pull down list. The system remained in use for ten years at various sites before being overtaken by newer technology and a single networked database system, so I know the pitfalls that await the unwary.

Either redesign the database with validated data fields and look up tables, or go through each record and apply an editing standard to free text records. If, for example, a name can appear spelled in one of several ways, e.g. with or without spaces, or with a hyphen or apostrophe, then adopt a standard such as adding the name at the end of the data field spelled all in capitals with no spaces.

Similarly inserting markers such as capitalised double or triple letter groups to designate certain types of records can be useful, E.g. JJ, QQ, YYY.

Try to standardise the data, then you can use standardised searches and expect to be successful in quickly finding what you are looking for most of the time.

Database design is something of a black art. You need to consider all the possibilities and likelyhood of needing extra data entry fields before you even begin.

Whichever path you chose, you have a lot of work ahead. Good luck.
G0ULI is offline