It will depend upon which state you are building in, but generally that answer is no, that is not true.
For example, in Oklahoma, if you are not in city limits, you come under the jurisdiction of the county; if the county does not provide code enforcement services, then you will come under the jurisdiction of the State Fire Marshal. However, because of limited funding and manpower, the State Fire Marshal’s office does not review all kinds of projects. Regardless, state law requires compliance with the building code that is adopted by the state.
In addition, you should also consider the moral duty to provide a safe environment to the occupants of your building, and beyond that, the liability you may face should someone be injured or die due to an unsafe building.