It's very easy to set up a sole-proprietor company. The part you usually register is the business name, to make sure nobody else was using it first. You need to set up your bookkeeping so that you can get the numbers straight, you may need to get a tax ID number if you are dealing with sales tax, you need an employer ID number if you are going to have employees.
Your other choice is incorporation, which takes a bit more time and money. There are reasons pro & con for incorporation vs sole-proprietor, which you can look up on the internet or any basic start-your-own-business book.
Note that if all of your business income comes from one client, the IRS will consider you an employee and disallow all of your deductions. That is, if they find you. It's a hot topic with them right now, so they're looking for that.
How hard would it be for you to get another client or two?