The average total cost (ATC) and average variable cost (AVC) curves are "\\cup" - shaped because of diminishing marginal returns (the law of variable proportions).
In the short run, capital is fixed whereas labour is variable. Increasing labour to the fixed capital will, after a certain point, result in a decrease in productivity. The decrease in productivity means an increase in average costs.
Initially, ATC and AVC decrease due to increasing marginal returns. Increasing labour to the fixed capital results in an increase in productivity. The rate of increase in output exceeds the rate of increase in variable inputs - labour. Thus, average costs decline as output increases. Technically, total cost increases at a decreasing rate; marginal cost falls. As it falls, marginal cost, it pulls AVC and ATC downwards.
Secondly, AVC and ATC increase due to diminishing marginal returns to labour. Further increases in the variable input to the fixed capital results in output increasing at a lower rate than an increase in variable factor inputs. Average costs thus start to increase. At this point, the fixed capital becomes overutilized and strained. Technically, total cost start to increase at an increasing rate; marginal cost increases. As marginal cost increases, it pulls AVC and ATC upwards.
Thus, due to the law of variable proportions (diminishing marginal returns), the AVC and ATC curves are "\\cup" - shaped. They decrease due to increasing marginal returns to the variable factor, reaches minimum as diminishing marginal returns set in, and eventually increase due to diminishing marginal returns.