Problems with JDBC

I am trying to make a word game and obviously I need a word list to check the correctness of the words that people enter, so I found a .txt file containing all the words I could ever want and wrote some code to have the .txt file loaded line by line into a database using Java Database Connectivity, but now I am trying to query the database to see if it can retrieve information from the database. If it isn’t able to retrieve anything, then it will tell the user that it is not a word, but if it is able to retrieve info, that means the word is in the database and therefore is a real word… My program doesn’t work though, it keeps giving me the following error when I run it at the point in time that it tries to query the database…

ERROR: java.sql.SQLException: [Microsoft][ODBC Microsoft Access Driver] Too few parameters. Expected 1.

Here is the code I am using, any ideas?

/*
 * The purpose of this class is to test that I can connect to the database
 * I created with WordDBMaker.java and WordDBLoader.java and query the database
 * to check that words the user enters are in the database and therefore valid
 * words.
 */

/**
 * @author Josh Branchaud
 * @version 12/30/2008
 */

//*************************************
//***** Imported Classes **************
//*************************************
// added for reading user input from the console
import java.util.Scanner;

// added for connecting to and querying the database
import java.sql.DriverManager;
import java.sql.Connection;
import java.sql.Statement;
import java.sql.SQLException;
import java.sql.ResultSet;

public class WordDBTester
{
    // create a volatile boolean variable for knowing when to end execution
    public volatile boolean endExecution = false;

    // the main method for this class
    public static void main(String[] args)
    {
        WordDBTester WDBT = new WordDBTester();

        // Scanner called in for reading user input from the console
        Scanner in = new Scanner(System.in);

        // try/catch block for connecting to the database and querying it.
        try
        {
            // specifying the driver we want to use for database connection
            Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");

            // String variable holding the location of the database
            String DBLocation = "jdbc:odbc:WordList1";

            // connect to the database found at DBLocation
            Connection DBConnection = DriverManager.getConnection(DBLocation);

            // system message for confirmation of connection to the database
            System.out.println("Successfully connected to " + DBLocation);

            Statement s1 = DBConnection.createStatement();

            while(!(WDBT.endExecution))
            {
                System.out.println("Type 'quit!' if you would like to exit, otherwise");
                System.out.print("enter the word you would like to check: ");

                String inputWord = "";

                inputWord = in.nextLine();

                System.out.println("");

                if(inputWord.compareTo("quit!") == 0)
                {
                    WDBT.endExecution = true;
                    break;
                }

                System.out.println("The word you are checking for is " + inputWord);

                String wordQuery = "SELECT * FROM WordList1 WHERE Word = " + inputWord;

                System.out.println("The query looks like " + wordQuery);

                ResultSet rs = s1.executeQuery(wordQuery);

                int resultCount = 0;
                String thisWord = "";
                int pointValue = 0;

                while(rs.next())
                {
                    resultCount++;

                    thisWord = rs.getString("Word");
                    pointValue = rs.getInt("Points");
                }

                if(resultCount == 0)
                {
                    System.out.println("Sorry, but " + inputWord + " is not a word.");
                }
                else
                {
                    System.out.println(thisWord + " is a word and it is worth " + pointValue + " points.");
                }
            }

            // disconnect from the database
            DBConnection.close();

            // confirmation message of the disconnection from the database
            System.out.println("Disconnection successful!");
        }
        catch(ClassNotFoundException CNFError)
        {
            System.out.println("ERROR: " + CNFError);
        }
        catch(SQLException SQLError)
        {
            System.out.println("ERROR: " + SQLError);
        }
        catch(Exception Error)
        {
            System.out.println("ERROR: " + Error);
        }
    }
}

You should use MySQL -> www.dev.mysql.com

Its way better than access, and PHP supports it too :wink:

But for your simple word game, it would be better off using a encrypted file.

Anyway, this is how I used MySQL in my server:


           String driver = "com.mysql.jdbc.Driver";
           String url = "jdbc:mysql://localhost:3306/renoria";
           String username = "root";
           String password = "***************";
            try {
                Class.forName(driver); // touch the MySQL driver
            } catch (ClassNotFoundException e) {
                System.out.println("Database error:");
                e.printStackTrace();
            }
            try {
                Connection con = DriverManager.getConnection(url, username, password);
                allConnections.add(con);
                System.out.println("Database Connection Initiallized.");
                return con;
            } catch (SQLException e) {
                System.out.println("Database error:");
                e.printStackTrace();
                return null;
            }

Also, you must install the MySQL ODBC JDBC driver as a lib in your project.

I was wondering if mySQL might be a better way to go as far as databases go. Mostly, I went with Access because it was already on my machine. Where do I find a driver for that? Also, when you say I need to include it in my project library, do you mean the that the sql driver is a jar file and I need to add it to the compile time libraries?

How big is your word list? Generating and connecting to an external database just to check if a word is on the list seems like overkill if you ask me - can’t you just store the thing in memory using a HashSet or something like that?

Well, the list has a little over 200,000 words, I am trying to get a slightly more extensive one though. Anyway, I considered a hashmap, but I don’t really know anything about Hashsets. What do you suggest instead of a database, it seems to be working okay so far?

Personally… I would go with an SQL database. 200k is a TON of memory to be holding in a HashMap. Plus, with SQL, you have already have wildcard searching and sorting done for you.

Before I go on, let me just tell you what’s wrong with you’re query:


String wordQuery = "SELECT * FROM WordList1 WHERE Word = " + inputWord;

If I’m looking for the word “dog” … here is what your query looks like to the DB:


SELECT * FROM WordList1 WHERE Word = dog

The word “dog” needs to be surrounded by quotes.

Going back to the DB discussion…

I disagree with Renoria… you should not use MySQL for this (unless the code you posted is for a server app). MySQL is a great production database for a server to use, but it’s unreasonable to assume every client that downloads your application will have MySQL installed locally. Using an Access database is an okay alternative - since everything is local - but I don’t know if that will work on non-Windows platforms.

In my opinion, the best option would be to use SQLite. SQLiteJDBC gives you the option to load a platform-specfific native DB driver, or pure-Java.

This way, you can let SQLite search the giant list of words for you, and you don’t have to worry about JDBC driver compatibilities (because you can deploy SQLiteJDBC with your app).

I was looking at SQLLite, do I do the same coding as I have been doing and this is just a different driver in order to access my database?

Also, I have never done a project like this before and I am wondering once I have my word game complete (probably just with GUI) and a word list that I am happy with (stored in a database), how do I make this accessible to anyone? I am thinking of making it into an applet that will be on my website, do i need to have my database stored somewhere so that anyone from any computer can play the game and the game will be able to access the database?

Well, if you’re using something like Access or SQLite, the DB is just a local file, so it’s no different than accessing any other file.

Question… for your word database, do you ONLY want to check if a word exists? Nothing more? If so, you can probably get away with reading a plain text file that contains all the words.

200k is a lot of words, but you can use a simple index strategy (All ‘a’ words go into a.txt, all ‘b’ words go into b.txt … to reduce how much data you scan)

I wrote you an example you can build off of. The words.txt contains all the words. To build the index, run this command:


java Index

This will create an index folder, and you’ll see the a.txt, b.txt, c.txt… etc. files in there.

Then run this:


java Example

As you can see from the Example… the speed of the word lookup doesn’t take long at all.

So, building off this example… all you have to do is add your words to words.txt, then rebuild the index. No database needed.

Ooh, I considered that idea a while ago, but didn’t go anywhere with it. Would that work in the context of a web applet?

Sure, all you have to do is package your applet as a JAR, and include the index folder inside that JAR. Then to access the files you just call getClass().getResource(…)

Example: http://woogley.net/misc/WordList/applet.zip

actually, once its a JAR, you cannot use getResource() anymore. You MUST use getResourceAsStream().

@Renoria
Thats a misconception, you can use getResource(), but you can’t use getResource().getFile().

If you ran the applet demo I posted, you would’ve immediately found out you are incorrect. I mean, dude, it is literally right in front of you.

Okay, that makes sense.

So it would really be faster to read through a file every time I want to check for a word rather than connect to a database at the beginning of the game and have O(1) search time the rest of the game?

Also, I had planned to use the database for a few extra things, for instance, there were some extra columns with the word list the contained how many points each word was worth as well as some other statistics about each word, this was all calculated at the time of populating the database, that way I wouldn’t have to do it each time the user entered a word in actual game play.

Well, this is debatable. SQL’s search is probably faster on the millisecond level, but both options determine if a word exists in less than a second. It’s instantaneous to the end user.

Normally I use SQLite whenever possible, since it has so much built in. But for an applet, getting JDBC to work consistently means you’re likely going to have to sign your applet and include native libraries. SQLite does offer a pure-Java option that wouldn’t require signging, but it is very slow.

So… given the fact that you’re using an applet, I think you would save yourself some trouble by staying away from using a database.

It sounds like you’re already arriving at a reasonable solution; that said, it’s probably worth pointing out that a HashSet can handle 200k elements in memory without much trouble:


import java.util.HashSet;
import java.util.Random;

public class HashSetStressTest {

	/**
	 * Stress test java.util.HashSet with random strings inserted.
	 */
	public static void main(String[] args) {
		final int mapSize = 200000;
		System.out.println("Memory usage before HashSet creation: "+(Runtime.getRuntime().totalMemory()-Runtime.getRuntime().freeMemory()));
		HashSet<String> map = new HashSet<String>(mapSize);
		for (int i=0; i<mapSize; ++i) {
			map.add(randomString());
		}
		System.out.println(mapSize + " element HashSet created.");
		long memoryUsed = (Runtime.getRuntime().totalMemory()-Runtime.getRuntime().freeMemory());
		long tBegin = System.nanoTime();
		int foundCount = 0;
		for (int i=0; i<mapSize*100; ++i) {
			//System.out.println("Test "+i+": "+map.contains(randomString()));
			boolean found = map.contains(randomString());
			if (found) foundCount++;
		}
		long tEnd = System.nanoTime();
		System.out.println("Found "+foundCount+" random elements.");
		System.out.println("Done checking "+(100*mapSize)+" random elements.");
		System.out.println("Memory usage with HashSet in memory: "+memoryUsed);
		System.out.println("Took "+((tEnd-tBegin)/1000000000) + " seconds total.");
		double timePerElement = ((tEnd-tBegin)) / (100*mapSize);
		System.out.println("Nanos per check: "+timePerElement);
	}

	/** Generate a random 5 letter string. */
	static private String randomString() {
		StringBuilder s = new StringBuilder();
		Random rand = new Random();
		for (int i=0; i<5; ++i) {
			int myInt = rand.nextInt(25);
			char letter = (char)('a' + myInt);
			s.append(letter);
		}
		return s.toString();
	}
	
}

My results (Apple’s 1.5 client JVM):


Memory usage before HashSet creation: 441304
200000 element HashSet created.
Found 404694 random elements.
Done checking 20000000 random elements.
Memory usage with HashSet in memory: 17971792
Took 16 seconds total.
Nanos per check: 836.0

This is not meant to be an accurate benchmark or anything (in particular, those timings are probably overwhelmed by the random string generation), but it serves as a proof of concept: the point is, even with a 200,000 element HashSet, and wasting a lot of time generating strings, the lookup time is completely and utterly negligible (I can do 20 million checks in less than a minute), and the memory usage is about 20 megabytes - not exactly trim, but not overwhelming, either.

Not saying that this is necessarily the solution for you (I didn’t realize you were talking about such a massive word list, I thought you were talking maybe 30k), just figured it’s worth pointing out…I’d say go with a DB if you need the searching and wildcard features that it gives you, as woogley suggests; if you just need to check if something’s on the list or not, there are a lot of solutions that don’t involve extra dependencies, and to be honest, any of them will do just fine and you won’t notice any difference performance-wise.

Actually the hashmap size in memory should be rather small, it’s the words themselfs that take up the most space. Anyway 20mb is peanuts on any modern pc. if it’s really a concern you can sacrifice speed for memory: just stream the file. The database is going to occour overhead as well. I can’t see you getting that back by the extra optimalsations a database would bring. Esp since in this particular case most of the high impact optmalsations can be applied to the file based aproach and should be reasonable trivial to implement. As a bonus you can easilly add words by editing a file.

If I do something like a hashmap or .txt files is the best way of checking to do a String.compareTo()… or is there a different/better way?

If you have some sort of collection you can use contains() given that it might not be fast enough (not likely) you can look at building a tree out of the words by using chars for nodes. Though just using the first letter(or two) as a key in a map might already be a ‘good enough’.

As for using the text file, you can either stream the whole file (which is probably fast enough) but… you can optimise by making sure the entries is the file are sorted or are sorted in a new file and use that and map by the first letter and skip to that byte(/char) or sort into different (temp?)files a.db b.db c.db and only read that file… or combination of the two.

If you do it in-memory and the only thing you need to do is check if a word is in the list, HashSet is probably the way to go - again, if you need any other functionality, it might not suffice. A HashSet just contains a bunch of objects and the only really useful thing it does is allow you to use contains() to see if an element is there or not. If you can fit your list in a reasonable amount of memory, then contains() will not cause any perceptible speed problems at all unless you’re literally calling it thousands of times per frame.

Example:


HashSet<String> mySet = new HashSet<String>();
mySet.add("Hello");
mySet.add("Goodbye");
boolean hasHello = mySet.contains("Hello"); //true
boolean hasGoodDay = mySet.contains("Good Day"); //false