Banks (Bancos), are like banks in any country.
Caja, Is more like a Savings and loan association (USA).
Caja, is owned by the Municipality, Banks are owned by shareholders.
Caja, is principally in the business of "Micro finance" and make loans as small as S/.250 but can also charge interest up to 52% per annum.
Peruvian banks are rated safer then Peruvian Caja's. Different Municipal Caja's are safer then others.
Peruvian Gov't insures deposits up to S/.85,000 (per institution) for both.
Rates for savings are higher in Caja and Loans are cheaper in Banks.