I can`t see a problem other than the extra weight and I`d say make the fire box bigger than suggested to allow for the brick thickness. now don`t forget the pic`s of the build!
It will hold heat a LOT longer, and will also take longer to get them to the point were they will hold the temp. Your fire box will be up to temp way before your fire brick. There's different types of refractory and they all have different thermal properties. It would help to research them. Good luck and let us know what you come up with.