The three material types vary in more than just health. Wood structures have a max of 200 health while brick is set at 300. Metal trumps them all with a max of 400 health.
During combat, wood is probably the best material to use. Wood builds the fastest, meaning that when it is initially placed down, it will have more base health than the other two materials.
Save your brick and metal for your fort building when you have the time to do so.
