I can share some of my experience with you since I've built a large custom house, and have completed plans for another one.
There is good advice from others here, but some which you may want to take with a grain of salt.
I don't think you need to have a net worth of $30 million to build a $1.5 million home. Those numbers seem way off.
Ilya is correct that contractors and subcontractors are really hurting right now, and you can strike some screaming deals on their labor. I have noticed that material costs are fluctuating from the past few years, depending on what it is. For instance, rebar went from low to very high and has leveled off again.
You may be able to get away with less than $200/ sq ft, depending on the fit and finish. However, be aware that you can easily push double that if you go really extravagant, so watch what finish you have throughout. Do you really need triple-stage crown molding in every room, or can you get away with just some decorative touches in just the kitchen, living room, and master bedroom and bath? You need to determine this into your budget, preferably when working with your architect. You can also save some money by doing some stuff yourself (such as pulling CAT6 and RG6 for whole house distribution, or painting).
The challenge, which we have encountered for our project as well, is that banks are not willing to give out construction loans readily in this economy, so a huge hurdle is finding a lender who has reasonable requirements. Your best bet would be a local bank, as the national banks guidelines are very stringent. At least 30%+ of your skin in the game, plus exceptional FICO scores, and a cost to value ratio of below 50%. Terms are about 12 months, with possible extensions ($$ believe me, I know) up to 18 months. Roll-over conversion from construction to perm but you may have to lock in the rate now (which could be a good thing since they are very low). Is your $1.5 million cost-to-build home actually going to be worth considerably more once it is built? How much sweat equity will you have versus just buying a home? Your bank will be very very interested in this, because if it is worth just $1.5 million when completed, they will not take that risk, unless you pony up your own money and bring down that final loan amount to a ratio less than 70% (they typically want that ratio down below 50%).
In terms of soft costs, don't neglect the other fees, such as school fees. Permit fees can blindside you if you are not prepared, and you would be shocked how much school fees could be (we ended up paying many tens of thousands of dollars, and we didn't even have a child at the time). In all, we paid significantly over $100k for all the miscellaneous city and county fees, and this does not include architectural plans, structural plans, soils engineer, civil engineer, etc which adds up to large $$. Of course this varies based on where you are located, but I see you are in CA, so I would assume that you'll probably encounter something similar.
You will be shocked at how long things will take. The architectural plans could take many months, and if you are required to have it pass an architectural design review committee for your community in addition to your city planner, then it can really drag on. So add 4 months (realistically 6 months to a year) for the plans to be completed and approved. This is in conjuction with the structural engineer doing his plans. Then you have to have an energy guy calculate numbers for the home as well. This is before you can get the loan, since banks will not lend unless the plans are fully approved by the city, so the money for all the plans will come out of your own pocket. You need to bid out to contractors even before the loan, since they need to give you numbers for cost so you can submit that to the bank (get at least three bids). It will take them about a month to get the estimates from their subs, and then get it back to you. The loan process itself can take months. When you finally do get the money, if it is possible that you can wait, you should try to plan it around the seasons, as ideally you want to start construction and build according to schedule so that you have the roof up before it gets rainy. There is nothing more miserable than having all your framing and your crawlspace/basement/concrete pad soaked and leading to uneven walls and sometimes uneven floors from the curving of the wood studs.
Unexpected costs will be in the form of change orders. Contractors will rack up the bill with changes you want to make in the middle of construction, so you want to do your best to stick with the plans, or incorporate changes in the design stage with your architect, rather than with your contractor. It is far cheaper to do so.
Landscaping and hardscape can be very expensive as well, and if your community or city requires you to have it in place before moving in, then you will be delayed once again in living there (again speaking from experience). All in all, we bought our property in 2003, and didn't move in until late 2006... almost 3-1/2 years.
A few guys mentioned the stress on a marriage. It definitely does take a toll, so be prepared to have one of you make executive decisions.
Do not take this lightly. I would start making appointments with banks now to discuss your situation and see if you can even qualify for a construction loan before purchasing the property. Do your due diligence before making a rash decision about purchasing a lot. Otherwise, you will be stuck with bare land and a high interest lot loan which is just tying up your money.
Good luck.
