Primary keys must be 2 things - Not null (meaning noy empty) and unique in the table. For example, let's say you are designing a database for a hardware store inventory.
First, you'd want a table with perhaps the part number/stock number, item description, wholesale price, and retail price. The part number would be your primary key, since no two items would have the same part number.
So now let's say you want to track how many items are sold each month. You'd want a table to track those sales of items. In this table you'd want the same part number field, and perhaps the date and time of the transaction. So as the cashier scans or keys in the item number in the register, you can have it keep track of these items.
So for the month of January you sold 100 hammers and 50 screwdrivers. Your transaction sales table would contain 150 rows or entries for those items. Although this table is only storing the part number and date of what was sold, it is easy to generate a report with the full item descriptions since the part number is linked back to your main part table. This case is called a one-to-many relationship.
So what's the advantage of foreign keys then? Well they are handy as you can set up constraints on them to insure data integrity. Perhaps the hammer isn't scanning properly, so the cashier keys in the part number. But if she fat fingers it and mistypes the number, the database can check the foreign key against the parts table and not allow that row to be inserted into the table since that part number doesn't exist.
Also, depending on the database software, you can perform cascade updates and deletes on relationships. Say you want to update the part number for the hammer. But you've already recorded 100 hammers sold in the transaction table. With a cascade update, if you change it in the parts table it will automatically change all the corresponding entries in the transaction table to match. Very handy. However, not all database software does this (including some big iron ones) and you need to rely on triggers. But that is way out there.
Also, all tables should have a primary key. In the example above, it may seem the transaction table would pose a problem for choosing a primary key. However, there is a handy feature called an Identity or autonumber field type. You can set them to start out at any number and increment automatically. That way, you can let the database software do the work of assigning a unique id to each row in the table.
All this was probably as clear as mud

.