A novel extension of residual learning is presented for deep networks which effectively improves the robustness of the learned representation. The method integrates a plug-and-play module， that is, a grouped convolutional encoder-decoder， as additional shortcuts to the original residual architecture. Due to the down-sampling in encoder stage， the decoder modules are driven to produce focally activated feature maps， which highlights the most discriminative regions of input images， and imposes local enhancement on input features through element-wise addition. For efficient model design， we exploit lightweight counterparts by removing part channels of residual mappings， without showing obvious performance degradation. We obtain consistent accuracy gain for various residual architectures with comparable or even lower model complexity.