Since you mentioned that your entire site isn't in Japanese, in addition to what other people specified, I would now recommend using the CSS unicode-range selector. That allows you to list a web font declaration with a range of Unicode code points and most modern browsers now implement the optimization where it only downloads the font when text on the page contains a character in that range.

I tossed together an example several years ago:

http://chris.improbable.org/experiments/browser/webfonts/uni...

The combined example on http://chris.improbable.org/experiments/browser/webfonts/uni... has CSS which looks like this:

    @font-face {
        font-family: NotoSansCombined;
        src: local("NotoSans"), local("Noto Sans"), url(NotoSans-Regular.woff) format("woff");
    }

    @font-face {
        font-family: NotoSansCombined;
        src: local("NotoSansArmenian"), local("Noto Sans Armenian"), url(NotoSansArmenian-Regular.woff) format("woff");
        unicode-range: U+530-58F, U+FB13-FB17;
    }

    @font-face {
        font-family: NotoSansCombined;
        src: local("NotoSansBengali"), local("Noto Sans Bengali"), url(NotoSansBengali-Regular.woff) format("woff");
        unicode-range: U+AD, U+D7, U+F7, U+964-965, U+981-983, U+985-98C, U+98F-990, U+993-9A8, U+9AA-9B0, U+9B2, U+9B6-9B9, U+9BC-9C4, U+9C7-9C8, U+9CB-9CE, U+9D7, U+9DC-9DD, U+9DF-9E3, U+9E6-9FB, U+200B-200D, U+2013-2014, U+2018-2019, U+201C-201D, U+2026, U+20B9, U+2212, U+25CC;
    }

    @font-face {
        font-family: NotoSansCombined;
        src: local("NotoSansCherokee"), local("Noto Sans Cherokee"), url(NotoSansCherokee-Regular.woff) format("woff");
        unicode-range: U+13A0-13F4;
    }

In modern versions of Chrome, Firefox, and Safari you can see that only downloads a few of the many language files available. This also works really nicely with the traditional CSS font stacks if your design goals allow you to specify system fonts which will work for many users so you can have a few local fonts specified before the downloadable one.