512 bytes of data.
+What: /sys/block/<disk>/integrity/device_is_integrity_capable
+Date: July 2014
+Contact: Martin K. Petersen <martin.petersen@oracle.com>
+Description:
+ Indicates whether a storage device is capable of storing
+ integrity metadata. Set if the device is T10 PI-capable.
+
+
What: /sys/block/<disk>/integrity/write_generate
Date: June 2008
Contact: Martin K. Petersen <martin.petersen@oracle.com>
4.1 BIO
The data integrity patches add a new field to struct bio when
-CONFIG_BLK_DEV_INTEGRITY is enabled. bio->bi_integrity is a pointer
-to a struct bip which contains the bio integrity payload. Essentially
-a bip is a trimmed down struct bio which holds a bio_vec containing
-the integrity metadata and the required housekeeping information (bvec
-pool, vector count, etc.)
+CONFIG_BLK_DEV_INTEGRITY is enabled. bio_integrity(bio) returns a
+pointer to a struct bip which contains the bio integrity payload.
+Essentially a bip is a trimmed down struct bio which holds a bio_vec
+containing the integrity metadata and the required housekeeping
+information (bvec pool, vector count, etc.)
A kernel subsystem can enable data integrity protection on a bio by
calling bio_integrity_alloc(bio). This will allocate and attach the
supported by the block device.
- int bdev_integrity_enabled(block_device, int rw);
-
- bdev_integrity_enabled() will return 1 if the block device
- supports integrity metadata transfer for the data direction
- specified in 'rw'.
-
- bdev_integrity_enabled() honors the write_generate and
- read_verify flags in sysfs and will respond accordingly.
-
-
int bio_integrity_prep(bio);
To generate IMD for WRITE and to set up buffers for READ, the
bio_integrity_enabled() returned 1.
- int bio_integrity_tag_size(bio);
-
- If the filesystem wants to use the application tag space it will
- first have to find out how much storage space is available.
- Because tag space is generally limited (usually 2 bytes per
- sector regardless of sector size), the integrity framework
- supports interleaving the information between the sectors in an
- I/O.
-
- Filesystems can call bio_integrity_tag_size(bio) to find out how
- many bytes of storage are available for that particular bio.
-
- Another option is bdev_get_tag_size(block_device) which will
- return the number of available bytes per hardware sector.
-
-
- int bio_integrity_set_tag(bio, void *tag_buf, len);
-
- After a successful return from bio_integrity_prep(),
- bio_integrity_set_tag() can be used to attach an opaque tag
- buffer to a bio. Obviously this only makes sense if the I/O is
- a WRITE.
-
-
- int bio_integrity_get_tag(bio, void *tag_buf, len);
-
- Similarly, at READ I/O completion time the filesystem can
- retrieve the tag buffer using bio_integrity_get_tag().
-
-
5.3 PASSING EXISTING INTEGRITY METADATA
Filesystems that either generate their own integrity metadata or
.name = "STANDARDSBODY-TYPE-VARIANT-CSUM",
.generate_fn = my_generate_fn,
.verify_fn = my_verify_fn,
- .get_tag_fn = my_get_tag_fn,
- .set_tag_fn = my_set_tag_fn,
.tuple_size = sizeof(struct my_tuple_size),
.tag_size = <tag bytes per hw sector>,
};
are available per hardware sector. For DIF this is either 2 or
0 depending on the value of the Control Mode Page ATO bit.
- See 6.2 for a description of get_tag_fn and set_tag_fn.
-
----------------------------------------------------------------------
2007-12-24 Martin K. Petersen <martin.petersen@oracle.com>
- reg : should specify the address and size used for NFC command registers,
NFC registers and NFC Sram. NFC Sram address and size can be absent
if don't want to use it.
+ - clocks: phandle to the peripheral clock
- Optional properties:
- atmel,write-by-sram: boolean to enable NFC write by sram.
compatible = "atmel,sama5d3-nfc";
#address-cells = <1>;
#size-cells = <1>;
+ clocks = <&hsmc_clk>
reg = <
0x70000000 0x10000000 /* NFC Command Registers */
0xffffc000 0x00000070 /* NFC HSMC regs */
file systems on embedded devices.
- compatible : should contain the specific model of mtd chip(s)
- used, if known, followed by either "cfi-flash", "jedec-flash"
- or "mtd-ram".
+ used, if known, followed by either "cfi-flash", "jedec-flash",
+ "mtd-ram" or "mtd-rom".
- reg : Address range(s) of the mtd chip(s)
It's possible to (optionally) define multiple "reg" tuples so that
non-identical chips can be described in one node.
STMicroelectronics SoC DWMAC glue layer controller
+This file documents differences between the core properties in
+Documentation/devicetree/bindings/net/stmmac.txt
+and what is needed on STi platforms to program the stmmac glue logic.
+
The device node has following properties.
Required properties:
- - compatible : Can be "st,stih415-dwmac", "st,stih416-dwmac" or
- "st,stid127-dwmac".
- - reg : Offset of the glue configuration register map in system
+ - compatible : Can be "st,stih415-dwmac", "st,stih416-dwmac",
+ "st,stih407-dwmac", "st,stid127-dwmac".
+ - reg : Offset of the glue configuration register map in system
configuration regmap pointed by st,syscon property and size.
-
- - reg-names : Should be "sti-ethconf".
-
- - st,syscon : Should be phandle to system configuration node which
+ - st,syscon : Should be phandle to system configuration node which
encompases this glue registers.
+ - st,gmac_en: this is to enable the gmac into a dedicated sysctl control
+ register available on STiH407 SoC.
+ - sti-ethconf: this is the gmac glue logic register to enable the GMAC,
+ select among the different modes and program the clk retiming.
+ - pinctrl-0: pin-control for all the MII mode supported.
- - st,tx-retime-src: On STi Parts for Giga bit speeds, 125Mhz clocks can be
- wired up in from different sources. One via TXCLK pin and other via CLK_125
- pin. This wiring is totally board dependent. However the retiming glue
- logic should be configured accordingly. Possible values for this property
-
- "txclk" - if 125Mhz clock is wired up via txclk line.
- "clk_125" - if 125Mhz clock is wired up via clk_125 line.
-
- This property is only valid for Giga bit setup( GMII, RGMII), and it is
- un-used for non-giga bit (MII and RMII) setups. Also note that internal
- clockgen can not generate stable 125Mhz clock.
-
- - st,ext-phyclk: This boolean property indicates who is generating the clock
- for tx and rx. This property is only valid for RMII case where the clock can
- be generated from the MAC or PHY.
-
- - clock-names: should be "sti-ethclk".
- - clocks: Should point to ethernet clockgen which can generate phyclk.
-
+Optional properties:
+ - resets : phandle pointing to the system reset controller with correct
+ reset line index for ethernet reset.
+ - st,ext-phyclk: valid only for RMII where PHY can generate 50MHz clock or
+ MAC can generate it.
+ - st,tx-retime-src: This specifies which clk is wired up to the mac for
+ retimeing tx lines. This is totally board dependent and can take one of the
+ posssible values from "txclk", "clk_125" or "clkgen".
+ If not passed, the internal clock will be used by default.
+ - sti-ethclk: this is the phy clock.
+ - sti-clkconf: this is an extra sysconfig register, available in new SoCs,
+ to program the clk retiming.
+ - st,gmac_en: to enable the GMAC, this only is present in some SoCs; e.g.
+ STiH407.
Example:
-ethernet0: dwmac@fe810000 {
- device_type = "network";
- compatible = "st,stih416-dwmac", "snps,dwmac", "snps,dwmac-3.710";
- reg = <0xfe810000 0x8000>, <0x8bc 0x4>;
- reg-names = "stmmaceth", "sti-ethconf";
- interrupts = <0 133 0>, <0 134 0>, <0 135 0>;
- interrupt-names = "macirq", "eth_wake_irq", "eth_lpi";
- phy-mode = "mii";
+ethernet0: dwmac@9630000 {
+ device_type = "network";
+ status = "disabled";
+ compatible = "st,stih407-dwmac", "snps,dwmac", "snps,dwmac-3.710";
+ reg = <0x9630000 0x8000>, <0x80 0x4>;
+ reg-names = "stmmaceth", "sti-ethconf";
- st,syscon = <&syscfg_rear>;
+ st,syscon = <&syscfg_sbc_reg>;
+ st,gmac_en;
+ resets = <&softreset STIH407_ETH1_SOFTRESET>;
+ reset-names = "stmmaceth";
- snps,pbl = <32>;
+ interrupts = <GIC_SPI 98 IRQ_TYPE_NONE>,
+ <GIC_SPI 99 IRQ_TYPE_NONE>,
+ <GIC_SPI 100 IRQ_TYPE_NONE>;
+ interrupt-names = "macirq", "eth_wake_irq", "eth_lpi";
+
+ snps,pbl = <32>;
snps,mixed-burst;
- resets = <&softreset STIH416_ETH0_SOFTRESET>;
- reset-names = "stmmaceth";
- pinctrl-0 = <&pinctrl_mii0>;
- pinctrl-names = "default";
- clocks = <&CLK_S_GMAC0_PHY>;
- clock-names = "stmmaceth";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_rgmii1>;
+
+ clock-names = "stmmaceth", "sti-ethclk";
+ clocks = <&CLK_S_C0_FLEXGEN CLK_EXT2F_A9>,
+ <&CLK_S_C0_FLEXGEN CLK_ETH_PHY>;
};
F: arch/alpha/
ALTERA TRIPLE SPEED ETHERNET DRIVER
-M: Vince Bridgers <vbridgers2013@gmail.com>
+M: Vince Bridgers <vbridger@opensource.altera.com>
L: netdev@vger.kernel.org
L: nios2-dev@lists.rocketboards.org (moderated for non-subscribers)
S: Maintained
F: drivers/acpi/dock.c
DOCUMENTATION
-M: Jiri Kosina <jkosina@suse.cz>
+M: Jonathan Corbet <corbet@lwn.net>
L: linux-doc@vger.kernel.org
S: Maintained
F: Documentation/
X: Documentation/ABI/
X: Documentation/devicetree/
X: Documentation/[a-z][a-z]_[A-Z][A-Z]/
-T: git git://git.kernel.org/pub/scm/linux/kernel/git/jikos/doc.git
DOUBLETALK DRIVER
M: "James R. Van Zandt" <jrv@vanzandt.mv.com>
W: http://www.linux-mtd.infradead.org/
Q: http://patchwork.ozlabs.org/project/linux-mtd/list/
T: git git://git.infradead.org/linux-mtd.git
+T: git git://git.infradead.org/l2-mtd.git
S: Maintained
F: drivers/mtd/
F: include/linux/mtd/
break;
}
+ gpmc_nand_data->flash_bbt = of_get_nand_on_flash_bbt(child);
+
val = of_get_nand_bus_width(child);
if (val == 16)
gpmc_nand_data->devsize = NAND_BUSWIDTH_16;
status = "ok";
};
+&sgenet0 {
+ status = "ok";
+};
+
&xgenet {
status = "ok";
};
clock-output-names = "menetclk";
};
+ sge0clk: sge0clk@1f21c000 {
+ compatible = "apm,xgene-device-clock";
+ #clock-cells = <1>;
+ clocks = <&socplldiv2 0>;
+ reg = <0x0 0x1f21c000 0x0 0x1000>;
+ reg-names = "csr-reg";
+ csr-mask = <0x3>;
+ clock-output-names = "sge0clk";
+ };
+
xge0clk: xge0clk@1f61c000 {
compatible = "apm,xgene-device-clock";
#clock-cells = <1>;
};
};
+ sgenet0: ethernet@1f210000 {
+ compatible = "apm,xgene-enet";
+ status = "disabled";
+ reg = <0x0 0x1f210000 0x0 0x10000>,
+ <0x0 0x1f200000 0x0 0X10000>,
+ <0x0 0x1B000000 0x0 0X20000>;
+ reg-names = "enet_csr", "ring_csr", "ring_cmd";
+ interrupts = <0x0 0xA0 0x4>;
+ dma-coherent;
+ clocks = <&sge0clk 0>;
+ local-mac-address = [00 00 00 00 00 00];
+ phy-connection-type = "sgmii";
+ };
+
xgenet: ethernet@1f610000 {
compatible = "apm,xgene-enet";
status = "disabled";
297: wr %o5, FPRS_FEF, %fprs; \
298:
+#define VISEntryHalfFast(fail_label) \
+ rd %fprs, %o5; \
+ andcc %o5, FPRS_FEF, %g0; \
+ be,pt %icc, 297f; \
+ nop; \
+ ba,a,pt %xcc, fail_label; \
+297: wr %o5, FPRS_FEF, %fprs;
+
#define VISExitHalf \
wr %o5, 0, %fprs;
#endif
#endif
+#if !defined(EX_LD) && !defined(EX_ST)
+#define NON_USER_COPY
+#endif
+
#ifndef EX_LD
#define EX_LD(x) x
#endif
mov EX_RETVAL(%o3), %o0
.Llarge_src_unaligned:
+#ifdef NON_USER_COPY
+ VISEntryHalfFast(.Lmedium_vis_entry_fail)
+#else
+ VISEntryHalf
+#endif
andn %o2, 0x3f, %o4
sub %o2, %o4, %o2
- VISEntryHalf
alignaddr %o1, %g0, %g1
add %o1, %o4, %o1
EX_LD(LOAD(ldd, %g1 + 0x00, %f0))
nop
ba,a,pt %icc, .Lmedium_unaligned
+#ifdef NON_USER_COPY
+.Lmedium_vis_entry_fail:
+ or %o0, %o1, %g2
+#endif
.Lmedium:
LOAD(prefetch, %o1 + 0x40, #n_reads_strong)
andcc %g2, 0x7, %g0
int gs_ldt_reload_needed;
int fs_reload_needed;
u64 msr_host_bndcfgs;
+ unsigned long vmcs_host_cr4; /* May not match real cr4 */
} host_state;
struct {
int vm86_active;
u32 low32, high32;
unsigned long tmpl;
struct desc_ptr dt;
+ unsigned long cr4;
vmcs_writel(HOST_CR0, read_cr0() & ~X86_CR0_TS); /* 22.2.3 */
- vmcs_writel(HOST_CR4, read_cr4()); /* 22.2.3, 22.2.5 */
vmcs_writel(HOST_CR3, read_cr3()); /* 22.2.3 FIXME: shadow tables */
+ /* Save the most likely value for this task's CR4 in the VMCS. */
+ cr4 = read_cr4();
+ vmcs_writel(HOST_CR4, cr4); /* 22.2.3, 22.2.5 */
+ vmx->host_state.vmcs_host_cr4 = cr4;
+
vmcs_write16(HOST_CS_SELECTOR, __KERNEL_CS); /* 22.2.4 */
#ifdef CONFIG_X86_64
/*
static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu)
{
struct vcpu_vmx *vmx = to_vmx(vcpu);
- unsigned long debugctlmsr;
+ unsigned long debugctlmsr, cr4;
/* Record the guest's net vcpu time for enforced NMI injections. */
if (unlikely(!cpu_has_virtual_nmis() && vmx->soft_vnmi_blocked))
if (test_bit(VCPU_REGS_RIP, (unsigned long *)&vcpu->arch.regs_dirty))
vmcs_writel(GUEST_RIP, vcpu->arch.regs[VCPU_REGS_RIP]);
+ cr4 = read_cr4();
+ if (unlikely(cr4 != vmx->host_state.vmcs_host_cr4)) {
+ vmcs_writel(HOST_CR4, cr4);
+ vmx->host_state.vmcs_host_cr4 = cr4;
+ }
+
/* When single-stepping over STI and MOV SS, we must clear the
* corresponding interruptibility bits in the guest state. Otherwise
* vmentry fails as it then expects bit 14 (BS) in pending debug
bool seen_ld_abs;
};
+/* maximum number of bytes emitted while JITing one eBPF insn */
+#define BPF_MAX_INSN_SIZE 128
+#define BPF_INSN_SAFETY 64
+
static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image,
int oldproglen, struct jit_context *ctx)
{
struct bpf_insn *insn = bpf_prog->insnsi;
int insn_cnt = bpf_prog->len;
- u8 temp[64];
+ bool seen_ld_abs = ctx->seen_ld_abs | (oldproglen == 0);
+ u8 temp[BPF_MAX_INSN_SIZE + BPF_INSN_SAFETY];
int i;
int proglen = 0;
u8 *prog = temp;
EMIT2(0x31, 0xc0); /* xor eax, eax */
EMIT3(0x4D, 0x31, 0xED); /* xor r13, r13 */
- if (ctx->seen_ld_abs) {
+ if (seen_ld_abs) {
/* r9d : skb->len - skb->data_len (headlen)
* r10 : skb->data
*/
case BPF_JMP | BPF_CALL:
func = (u8 *) __bpf_call_base + imm32;
jmp_offset = func - (image + addrs[i]);
- if (ctx->seen_ld_abs) {
+ if (seen_ld_abs) {
EMIT2(0x41, 0x52); /* push %r10 */
EMIT2(0x41, 0x51); /* push %r9 */
/* need to adjust jmp offset, since
return -EINVAL;
}
EMIT1_off32(0xE8, jmp_offset);
- if (ctx->seen_ld_abs) {
+ if (seen_ld_abs) {
EMIT2(0x41, 0x59); /* pop %r9 */
EMIT2(0x41, 0x5A); /* pop %r10 */
}
goto common_load;
case BPF_LD | BPF_ABS | BPF_W:
func = CHOOSE_LOAD_FUNC(imm32, sk_load_word);
-common_load: ctx->seen_ld_abs = true;
+common_load:
+ ctx->seen_ld_abs = seen_ld_abs = true;
jmp_offset = func - (image + addrs[i]);
if (!func || !is_simm32(jmp_offset)) {
pr_err("unsupported bpf func %d addr %p image %p\n",
}
ilen = prog - temp;
+ if (ilen > BPF_MAX_INSN_SIZE) {
+ pr_err("bpf_jit_compile fatal insn size error\n");
+ return -EFAULT;
+ }
+
if (image) {
if (unlikely(proglen + ilen > oldproglen)) {
pr_err("bpf_jit_compile fatal error\n");
goto out;
}
if (image) {
- if (proglen != oldproglen)
+ if (proglen != oldproglen) {
pr_err("bpf_jit: proglen=%d != oldproglen=%d\n",
proglen, oldproglen);
+ goto out;
+ }
break;
}
if (proglen == oldproglen) {
config BLK_DEV_INTEGRITY
bool "Block layer data integrity support"
+ select CRC_T10DIF if BLK_DEV_INTEGRITY
---help---
Some storage devices allow extra information to be
stored/retrieved to help protect the data. The block layer
obj-$(CONFIG_IOSCHED_CFQ) += cfq-iosched.o
obj-$(CONFIG_BLOCK_COMPAT) += compat_ioctl.o
-obj-$(CONFIG_BLK_DEV_INTEGRITY) += blk-integrity.o
obj-$(CONFIG_BLK_CMDLINE_PARSER) += cmdline-parser.o
-obj-$(CONFIG_BLK_DEV_INTEGRITY) += bio-integrity.o
+obj-$(CONFIG_BLK_DEV_INTEGRITY) += bio-integrity.o blk-integrity.o t10-pi.o
+
bip->bip_slab = idx;
bip->bip_bio = bio;
bio->bi_integrity = bip;
+ bio->bi_rw |= REQ_INTEGRITY;
return bip;
err:
*/
void bio_integrity_free(struct bio *bio)
{
- struct bio_integrity_payload *bip = bio->bi_integrity;
+ struct bio_integrity_payload *bip = bio_integrity(bio);
struct bio_set *bs = bio->bi_pool;
- if (bip->bip_owns_buf)
- kfree(bip->bip_buf);
+ if (bip->bip_flags & BIP_BLOCK_INTEGRITY)
+ kfree(page_address(bip->bip_vec->bv_page) +
+ bip->bip_vec->bv_offset);
if (bs) {
if (bip->bip_slab != BIO_POOL_NONE)
int bio_integrity_add_page(struct bio *bio, struct page *page,
unsigned int len, unsigned int offset)
{
- struct bio_integrity_payload *bip = bio->bi_integrity;
+ struct bio_integrity_payload *bip = bio_integrity(bio);
struct bio_vec *iv;
if (bip->bip_vcnt >= bip->bip_max_vcnt) {
}
EXPORT_SYMBOL(bio_integrity_add_page);
-static int bdev_integrity_enabled(struct block_device *bdev, int rw)
-{
- struct blk_integrity *bi = bdev_get_integrity(bdev);
-
- if (bi == NULL)
- return 0;
-
- if (rw == READ && bi->verify_fn != NULL &&
- (bi->flags & INTEGRITY_FLAG_READ))
- return 1;
-
- if (rw == WRITE && bi->generate_fn != NULL &&
- (bi->flags & INTEGRITY_FLAG_WRITE))
- return 1;
-
- return 0;
-}
-
/**
* bio_integrity_enabled - Check whether integrity can be passed
* @bio: bio to check
* set prior to calling. The functions honors the write_generate and
* read_verify flags in sysfs.
*/
-int bio_integrity_enabled(struct bio *bio)
+bool bio_integrity_enabled(struct bio *bio)
{
+ struct blk_integrity *bi = bdev_get_integrity(bio->bi_bdev);
+
if (!bio_is_rw(bio))
- return 0;
+ return false;
/* Already protected? */
if (bio_integrity(bio))
- return 0;
+ return false;
+
+ if (bi == NULL)
+ return false;
+
+ if (bio_data_dir(bio) == READ && bi->verify_fn != NULL &&
+ (bi->flags & BLK_INTEGRITY_VERIFY))
+ return true;
+
+ if (bio_data_dir(bio) == WRITE && bi->generate_fn != NULL &&
+ (bi->flags & BLK_INTEGRITY_GENERATE))
+ return true;
- return bdev_integrity_enabled(bio->bi_bdev, bio_data_dir(bio));
+ return false;
}
EXPORT_SYMBOL(bio_integrity_enabled);
/**
- * bio_integrity_hw_sectors - Convert 512b sectors to hardware ditto
+ * bio_integrity_intervals - Return number of integrity intervals for a bio
* @bi: blk_integrity profile for device
- * @sectors: Number of 512 sectors to convert
+ * @sectors: Size of the bio in 512-byte sectors
*
* Description: The block layer calculates everything in 512 byte
- * sectors but integrity metadata is done in terms of the hardware
- * sector size of the storage device. Convert the block layer sectors
- * to physical sectors.
+ * sectors but integrity metadata is done in terms of the data integrity
+ * interval size of the storage device. Convert the block layer sectors
+ * to the appropriate number of integrity intervals.
*/
-static inline unsigned int bio_integrity_hw_sectors(struct blk_integrity *bi,
- unsigned int sectors)
+static inline unsigned int bio_integrity_intervals(struct blk_integrity *bi,
+ unsigned int sectors)
{
- /* At this point there are only 512b or 4096b DIF/EPP devices */
- if (bi->sector_size == 4096)
- return sectors >>= 3;
-
- return sectors;
+ return sectors >> (ilog2(bi->interval) - 9);
}
static inline unsigned int bio_integrity_bytes(struct blk_integrity *bi,
unsigned int sectors)
{
- return bio_integrity_hw_sectors(bi, sectors) * bi->tuple_size;
+ return bio_integrity_intervals(bi, sectors) * bi->tuple_size;
}
/**
- * bio_integrity_tag_size - Retrieve integrity tag space
- * @bio: bio to inspect
- *
- * Description: Returns the maximum number of tag bytes that can be
- * attached to this bio. Filesystems can use this to determine how
- * much metadata to attach to an I/O.
- */
-unsigned int bio_integrity_tag_size(struct bio *bio)
-{
- struct blk_integrity *bi = bdev_get_integrity(bio->bi_bdev);
-
- BUG_ON(bio->bi_iter.bi_size == 0);
-
- return bi->tag_size * (bio->bi_iter.bi_size / bi->sector_size);
-}
-EXPORT_SYMBOL(bio_integrity_tag_size);
-
-static int bio_integrity_tag(struct bio *bio, void *tag_buf, unsigned int len,
- int set)
-{
- struct bio_integrity_payload *bip = bio->bi_integrity;
- struct blk_integrity *bi = bdev_get_integrity(bio->bi_bdev);
- unsigned int nr_sectors;
-
- BUG_ON(bip->bip_buf == NULL);
-
- if (bi->tag_size == 0)
- return -1;
-
- nr_sectors = bio_integrity_hw_sectors(bi,
- DIV_ROUND_UP(len, bi->tag_size));
-
- if (nr_sectors * bi->tuple_size > bip->bip_iter.bi_size) {
- printk(KERN_ERR "%s: tag too big for bio: %u > %u\n", __func__,
- nr_sectors * bi->tuple_size, bip->bip_iter.bi_size);
- return -1;
- }
-
- if (set)
- bi->set_tag_fn(bip->bip_buf, tag_buf, nr_sectors);
- else
- bi->get_tag_fn(bip->bip_buf, tag_buf, nr_sectors);
-
- return 0;
-}
-
-/**
- * bio_integrity_set_tag - Attach a tag buffer to a bio
- * @bio: bio to attach buffer to
- * @tag_buf: Pointer to a buffer containing tag data
- * @len: Length of the included buffer
- *
- * Description: Use this function to tag a bio by leveraging the extra
- * space provided by devices formatted with integrity protection. The
- * size of the integrity buffer must be <= to the size reported by
- * bio_integrity_tag_size().
- */
-int bio_integrity_set_tag(struct bio *bio, void *tag_buf, unsigned int len)
-{
- BUG_ON(bio_data_dir(bio) != WRITE);
-
- return bio_integrity_tag(bio, tag_buf, len, 1);
-}
-EXPORT_SYMBOL(bio_integrity_set_tag);
-
-/**
- * bio_integrity_get_tag - Retrieve a tag buffer from a bio
- * @bio: bio to retrieve buffer from
- * @tag_buf: Pointer to a buffer for the tag data
- * @len: Length of the target buffer
- *
- * Description: Use this function to retrieve the tag buffer from a
- * completed I/O. The size of the integrity buffer must be <= to the
- * size reported by bio_integrity_tag_size().
- */
-int bio_integrity_get_tag(struct bio *bio, void *tag_buf, unsigned int len)
-{
- BUG_ON(bio_data_dir(bio) != READ);
-
- return bio_integrity_tag(bio, tag_buf, len, 0);
-}
-EXPORT_SYMBOL(bio_integrity_get_tag);
-
-/**
- * bio_integrity_generate_verify - Generate/verify integrity metadata for a bio
+ * bio_integrity_process - Process integrity metadata for a bio
* @bio: bio to generate/verify integrity metadata for
- * @operate: operate number, 1 for generate, 0 for verify
+ * @proc_fn: Pointer to the relevant processing function
*/
-static int bio_integrity_generate_verify(struct bio *bio, int operate)
+static int bio_integrity_process(struct bio *bio,
+ integrity_processing_fn *proc_fn)
{
struct blk_integrity *bi = bdev_get_integrity(bio->bi_bdev);
- struct blk_integrity_exchg bix;
+ struct blk_integrity_iter iter;
struct bio_vec *bv;
- sector_t sector;
- unsigned int sectors, ret = 0, i;
- void *prot_buf = bio->bi_integrity->bip_buf;
-
- if (operate)
- sector = bio->bi_iter.bi_sector;
- else
- sector = bio->bi_integrity->bip_iter.bi_sector;
+ struct bio_integrity_payload *bip = bio_integrity(bio);
+ unsigned int i, ret = 0;
+ void *prot_buf = page_address(bip->bip_vec->bv_page) +
+ bip->bip_vec->bv_offset;
- bix.disk_name = bio->bi_bdev->bd_disk->disk_name;
- bix.sector_size = bi->sector_size;
+ iter.disk_name = bio->bi_bdev->bd_disk->disk_name;
+ iter.interval = bi->interval;
+ iter.seed = bip_get_seed(bip);
+ iter.prot_buf = prot_buf;
bio_for_each_segment_all(bv, bio, i) {
void *kaddr = kmap_atomic(bv->bv_page);
- bix.data_buf = kaddr + bv->bv_offset;
- bix.data_size = bv->bv_len;
- bix.prot_buf = prot_buf;
- bix.sector = sector;
-
- if (operate)
- bi->generate_fn(&bix);
- else {
- ret = bi->verify_fn(&bix);
- if (ret) {
- kunmap_atomic(kaddr);
- return ret;
- }
- }
- sectors = bv->bv_len / bi->sector_size;
- sector += sectors;
- prot_buf += sectors * bi->tuple_size;
+ iter.data_buf = kaddr + bv->bv_offset;
+ iter.data_size = bv->bv_len;
+
+ ret = proc_fn(&iter);
+ if (ret) {
+ kunmap_atomic(kaddr);
+ return ret;
+ }
kunmap_atomic(kaddr);
}
return ret;
}
-/**
- * bio_integrity_generate - Generate integrity metadata for a bio
- * @bio: bio to generate integrity metadata for
- *
- * Description: Generates integrity metadata for a bio by calling the
- * block device's generation callback function. The bio must have a
- * bip attached with enough room to accommodate the generated
- * integrity metadata.
- */
-static void bio_integrity_generate(struct bio *bio)
-{
- bio_integrity_generate_verify(bio, 1);
-}
-
-static inline unsigned short blk_integrity_tuple_size(struct blk_integrity *bi)
-{
- if (bi)
- return bi->tuple_size;
-
- return 0;
-}
-
/**
* bio_integrity_prep - Prepare bio for integrity I/O
* @bio: bio to prepare
unsigned long start, end;
unsigned int len, nr_pages;
unsigned int bytes, offset, i;
- unsigned int sectors;
+ unsigned int intervals;
bi = bdev_get_integrity(bio->bi_bdev);
q = bdev_get_queue(bio->bi_bdev);
BUG_ON(bi == NULL);
BUG_ON(bio_integrity(bio));
- sectors = bio_integrity_hw_sectors(bi, bio_sectors(bio));
+ intervals = bio_integrity_intervals(bi, bio_sectors(bio));
/* Allocate kernel buffer for protection data */
- len = sectors * blk_integrity_tuple_size(bi);
+ len = intervals * bi->tuple_size;
buf = kmalloc(len, GFP_NOIO | q->bounce_gfp);
if (unlikely(buf == NULL)) {
printk(KERN_ERR "could not allocate integrity buffer\n");
return -EIO;
}
- bip->bip_owns_buf = 1;
- bip->bip_buf = buf;
+ bip->bip_flags |= BIP_BLOCK_INTEGRITY;
bip->bip_iter.bi_size = len;
- bip->bip_iter.bi_sector = bio->bi_iter.bi_sector;
+ bip_set_seed(bip, bio->bi_iter.bi_sector);
+
+ if (bi->flags & BLK_INTEGRITY_IP_CHECKSUM)
+ bip->bip_flags |= BIP_IP_CHECKSUM;
/* Map it */
offset = offset_in_page(buf);
/* Auto-generate integrity metadata if this is a write */
if (bio_data_dir(bio) == WRITE)
- bio_integrity_generate(bio);
+ bio_integrity_process(bio, bi->generate_fn);
return 0;
}
EXPORT_SYMBOL(bio_integrity_prep);
-/**
- * bio_integrity_verify - Verify integrity metadata for a bio
- * @bio: bio to verify
- *
- * Description: This function is called to verify the integrity of a
- * bio. The data in the bio io_vec is compared to the integrity
- * metadata returned by the HBA.
- */
-static int bio_integrity_verify(struct bio *bio)
-{
- return bio_integrity_generate_verify(bio, 0);
-}
-
/**
* bio_integrity_verify_fn - Integrity I/O completion worker
* @work: Work struct stored in bio to be verified
struct bio_integrity_payload *bip =
container_of(work, struct bio_integrity_payload, bip_work);
struct bio *bio = bip->bip_bio;
+ struct blk_integrity *bi = bdev_get_integrity(bio->bi_bdev);
int error;
- error = bio_integrity_verify(bio);
+ error = bio_integrity_process(bio, bi->verify_fn);
/* Restore original bio completion handler */
bio->bi_end_io = bip->bip_end_io;
*/
void bio_integrity_endio(struct bio *bio, int error)
{
- struct bio_integrity_payload *bip = bio->bi_integrity;
+ struct bio_integrity_payload *bip = bio_integrity(bio);
BUG_ON(bip->bip_bio != bio);
*/
void bio_integrity_advance(struct bio *bio, unsigned int bytes_done)
{
- struct bio_integrity_payload *bip = bio->bi_integrity;
+ struct bio_integrity_payload *bip = bio_integrity(bio);
struct blk_integrity *bi = bdev_get_integrity(bio->bi_bdev);
unsigned bytes = bio_integrity_bytes(bi, bytes_done >> 9);
void bio_integrity_trim(struct bio *bio, unsigned int offset,
unsigned int sectors)
{
- struct bio_integrity_payload *bip = bio->bi_integrity;
+ struct bio_integrity_payload *bip = bio_integrity(bio);
struct blk_integrity *bi = bdev_get_integrity(bio->bi_bdev);
bio_integrity_advance(bio, offset << 9);
int bio_integrity_clone(struct bio *bio, struct bio *bio_src,
gfp_t gfp_mask)
{
- struct bio_integrity_payload *bip_src = bio_src->bi_integrity;
+ struct bio_integrity_payload *bip_src = bio_integrity(bio_src);
struct bio_integrity_payload *bip;
BUG_ON(bip_src == NULL);
sizeof(struct bio_integrity_payload) +
sizeof(struct bio_vec) * BIP_INLINE_VECS,
0, SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL);
- if (!bip_slab)
- panic("Failed to create slab\n");
}
front_pad = 0;
inline_vecs = nr_iovecs;
} else {
+ /* should not use nobvec bioset for nr_iovecs > 0 */
+ if (WARN_ON_ONCE(!bs->bvec_pool && nr_iovecs > 0))
+ return NULL;
/*
* generic_make_request() converts recursion to iteration; this
* means if we're running beneath it, any bios we allocate and
}
EXPORT_SYMBOL(bioset_free);
-/**
- * bioset_create - Create a bio_set
- * @pool_size: Number of bio and bio_vecs to cache in the mempool
- * @front_pad: Number of bytes to allocate in front of the returned bio
- *
- * Description:
- * Set up a bio_set to be used with @bio_alloc_bioset. Allows the caller
- * to ask for a number of bytes to be allocated in front of the bio.
- * Front pad allocation is useful for embedding the bio inside
- * another structure, to avoid allocating extra data to go with the bio.
- * Note that the bio must be embedded at the END of that structure always,
- * or things will break badly.
- */
-struct bio_set *bioset_create(unsigned int pool_size, unsigned int front_pad)
+static struct bio_set *__bioset_create(unsigned int pool_size,
+ unsigned int front_pad,
+ bool create_bvec_pool)
{
unsigned int back_pad = BIO_INLINE_VECS * sizeof(struct bio_vec);
struct bio_set *bs;
if (!bs->bio_pool)
goto bad;
- bs->bvec_pool = biovec_create_pool(pool_size);
- if (!bs->bvec_pool)
- goto bad;
+ if (create_bvec_pool) {
+ bs->bvec_pool = biovec_create_pool(pool_size);
+ if (!bs->bvec_pool)
+ goto bad;
+ }
bs->rescue_workqueue = alloc_workqueue("bioset", WQ_MEM_RECLAIM, 0);
if (!bs->rescue_workqueue)
bioset_free(bs);
return NULL;
}
+
+/**
+ * bioset_create - Create a bio_set
+ * @pool_size: Number of bio and bio_vecs to cache in the mempool
+ * @front_pad: Number of bytes to allocate in front of the returned bio
+ *
+ * Description:
+ * Set up a bio_set to be used with @bio_alloc_bioset. Allows the caller
+ * to ask for a number of bytes to be allocated in front of the bio.
+ * Front pad allocation is useful for embedding the bio inside
+ * another structure, to avoid allocating extra data to go with the bio.
+ * Note that the bio must be embedded at the END of that structure always,
+ * or things will break badly.
+ */
+struct bio_set *bioset_create(unsigned int pool_size, unsigned int front_pad)
+{
+ return __bioset_create(pool_size, front_pad, true);
+}
EXPORT_SYMBOL(bioset_create);
+/**
+ * bioset_create_nobvec - Create a bio_set without bio_vec mempool
+ * @pool_size: Number of bio to cache in the mempool
+ * @front_pad: Number of bytes to allocate in front of the returned bio
+ *
+ * Description:
+ * Same functionality as bioset_create() except that mempool is not
+ * created for bio_vecs. Saving some memory for bio_clone_fast() users.
+ */
+struct bio_set *bioset_create_nobvec(unsigned int pool_size, unsigned int front_pad)
+{
+ return __bioset_create(pool_size, front_pad, false);
+}
+EXPORT_SYMBOL(bioset_create_nobvec);
+
#ifdef CONFIG_BLK_CGROUP
/**
* bio_associate_current - associate a bio with %current
static struct cgroup_subsys_state *
blkcg_css_alloc(struct cgroup_subsys_state *parent_css)
{
- static atomic64_t id_seq = ATOMIC64_INIT(0);
struct blkcg *blkcg;
if (!parent_css) {
blkcg->cfq_weight = CFQ_WEIGHT_DEFAULT;
blkcg->cfq_leaf_weight = CFQ_WEIGHT_DEFAULT;
- blkcg->id = atomic64_inc_return(&id_seq); /* root is 0, start from 1 */
done:
spin_lock_init(&blkcg->lock);
INIT_RADIX_TREE(&blkcg->blkg_tree, GFP_ATOMIC);
struct blkcg_gq *blkg_hint;
struct hlist_head blkg_list;
- /* for policies to test whether associated blkcg has changed */
- uint64_t id;
-
/* TODO: per-policy storage in blkcg */
unsigned int cfq_weight; /* belongs to cfq */
unsigned int cfq_leaf_weight;
* @bdev: device
*
* Locates the passed device's request queue and returns the address of its
- * backing_dev_info
- *
- * Will return NULL if the request queue cannot be located.
+ * backing_dev_info. This function can only be called if @bdev is opened
+ * and the return value is never NULL.
*/
struct backing_dev_info *blk_get_backing_dev_info(struct block_device *bdev)
{
- struct backing_dev_info *ret = NULL;
struct request_queue *q = bdev_get_queue(bdev);
- if (q)
- ret = &q->backing_dev_info;
- return ret;
+ return &q->backing_dev_info;
}
EXPORT_SYMBOL(blk_get_backing_dev_info);
* be drained. Check all the queues and counters.
*/
if (drain_all) {
+ struct blk_flush_queue *fq = blk_get_flush_queue(q, NULL);
drain |= !list_empty(&q->queue_head);
for (i = 0; i < 2; i++) {
drain |= q->nr_rqs[i];
drain |= q->in_flight[i];
- drain |= !list_empty(&q->flush_queue[i]);
+ if (fq)
+ drain |= !list_empty(&fq->flush_queue[i]);
}
}
#ifdef CONFIG_BLK_CGROUP
INIT_LIST_HEAD(&q->blkg_list);
#endif
- INIT_LIST_HEAD(&q->flush_queue[0]);
- INIT_LIST_HEAD(&q->flush_queue[1]);
- INIT_LIST_HEAD(&q->flush_data_in_flight);
INIT_DELAYED_WORK(&q->delay_work, blk_delay_work);
kobject_init(&q->kobj, &blk_queue_ktype);
if (!q)
return NULL;
- q->flush_rq = kzalloc(sizeof(struct request), GFP_KERNEL);
- if (!q->flush_rq)
+ q->fq = blk_alloc_flush_queue(q, NUMA_NO_NODE, 0);
+ if (!q->fq)
return NULL;
if (blk_init_rl(&q->root_rl, q, GFP_KERNEL))
return q;
fail:
- kfree(q->flush_rq);
+ blk_free_flush_queue(q->fq);
return NULL;
}
EXPORT_SYMBOL(blk_init_allocated_queue);
* pressure or if @q is dead.
*
* Must be called with @q->queue_lock held and,
- * Returns %NULL on failure, with @q->queue_lock held.
- * Returns !%NULL on success, with @q->queue_lock *not held*.
+ * Returns ERR_PTR on failure, with @q->queue_lock held.
+ * Returns request pointer on success, with @q->queue_lock *not held*.
*/
static struct request *__get_request(struct request_list *rl, int rw_flags,
struct bio *bio, gfp_t gfp_mask)
int may_queue;
if (unlikely(blk_queue_dying(q)))
- return NULL;
+ return ERR_PTR(-ENODEV);
may_queue = elv_may_queue(q, rw_flags);
if (may_queue == ELV_MQUEUE_NO)
* process is not a "batcher", and not
* exempted by the IO scheduler
*/
- return NULL;
+ return ERR_PTR(-ENOMEM);
}
}
}
* allocated with any setting of ->nr_requests
*/
if (rl->count[is_sync] >= (3 * q->nr_requests / 2))
- return NULL;
+ return ERR_PTR(-ENOMEM);
q->nr_rqs[is_sync]++;
rl->count[is_sync]++;
* shouldn't stall IO. Treat this request as !elvpriv. This will
* disturb iosched and blkcg but weird is bettern than dead.
*/
- printk_ratelimited(KERN_WARNING "%s: request aux data allocation failed, iosched may be disturbed\n",
- dev_name(q->backing_dev_info.dev));
+ printk_ratelimited(KERN_WARNING "%s: dev %s: request aux data allocation failed, iosched may be disturbed\n",
+ __func__, dev_name(q->backing_dev_info.dev));
rq->cmd_flags &= ~REQ_ELVPRIV;
rq->elv.icq = NULL;
rq_starved:
if (unlikely(rl->count[is_sync] == 0))
rl->starved[is_sync] = 1;
- return NULL;
+ return ERR_PTR(-ENOMEM);
}
/**
* function keeps retrying under memory pressure and fails iff @q is dead.
*
* Must be called with @q->queue_lock held and,
- * Returns %NULL on failure, with @q->queue_lock held.
- * Returns !%NULL on success, with @q->queue_lock *not held*.
+ * Returns ERR_PTR on failure, with @q->queue_lock held.
+ * Returns request pointer on success, with @q->queue_lock *not held*.
*/
static struct request *get_request(struct request_queue *q, int rw_flags,
struct bio *bio, gfp_t gfp_mask)
rl = blk_get_rl(q, bio); /* transferred to @rq on success */
retry:
rq = __get_request(rl, rw_flags, bio, gfp_mask);
- if (rq)
+ if (!IS_ERR(rq))
return rq;
if (!(gfp_mask & __GFP_WAIT) || unlikely(blk_queue_dying(q))) {
blk_put_rl(rl);
- return NULL;
+ return rq;
}
/* wait on @rl and retry */
spin_lock_irq(q->queue_lock);
rq = get_request(q, rw, NULL, gfp_mask);
- if (!rq)
+ if (IS_ERR(rq))
spin_unlock_irq(q->queue_lock);
/* q->queue_lock is unlocked at this point */
{
struct request *rq = blk_get_request(q, bio_data_dir(bio), gfp_mask);
- if (unlikely(!rq))
- return ERR_PTR(-ENOMEM);
+ if (IS_ERR(rq))
+ return rq;
blk_rq_set_block_pc(rq);
* Returns with the queue unlocked.
*/
req = get_request(q, rw_flags, bio, GFP_NOIO);
- if (unlikely(!req)) {
- bio_endio(bio, -ENODEV); /* @q is dead */
+ if (IS_ERR(req)) {
+ bio_endio(bio, PTR_ERR(req)); /* @q is dead */
goto out_unlock;
}
{
int total_bytes;
+ trace_block_rq_complete(req->q, req, nr_bytes);
+
if (!req->bio)
return false;
- trace_block_rq_complete(req->q, req, nr_bytes);
-
/*
* For fs requests, rq is just carrier of independent bio's
* and each partial completion should be handled separately.
error_type = "I/O";
break;
}
- printk_ratelimited(KERN_ERR "end_request: %s error, dev %s, sector %llu\n",
- error_type, req->rq_disk ?
+ printk_ratelimited(KERN_ERR "%s: %s error, dev %s, sector %llu\n",
+ __func__, error_type, req->rq_disk ?
req->rq_disk->disk_name : "?",
(unsigned long long)blk_rq_pos(req));
blk_rq_init(NULL, rq);
__rq_for_each_bio(bio_src, rq_src) {
- bio = bio_clone_bioset(bio_src, gfp_mask, bs);
+ bio = bio_clone_fast(bio_src, gfp_mask, bs);
if (!bio)
goto free_and_out;
*
* The actual execution of flush is double buffered. Whenever a request
* needs to execute PRE or POSTFLUSH, it queues at
- * q->flush_queue[q->flush_pending_idx]. Once certain criteria are met, a
+ * fq->flush_queue[fq->flush_pending_idx]. Once certain criteria are met, a
* flush is issued and the pending_idx is toggled. When the flush
* completes, all the requests which were pending are proceeded to the next
* step. This allows arbitrary merging of different types of FLUSH/FUA
FLUSH_PENDING_TIMEOUT = 5 * HZ,
};
-static bool blk_kick_flush(struct request_queue *q);
+static bool blk_kick_flush(struct request_queue *q,
+ struct blk_flush_queue *fq);
static unsigned int blk_flush_policy(unsigned int fflags, struct request *rq)
{
/* make @rq a normal request */
rq->cmd_flags &= ~REQ_FLUSH_SEQ;
rq->end_io = rq->flush.saved_end_io;
-
- blk_clear_rq_complete(rq);
}
static bool blk_flush_queue_rq(struct request *rq, bool add_front)
/**
* blk_flush_complete_seq - complete flush sequence
* @rq: FLUSH/FUA request being sequenced
+ * @fq: flush queue
* @seq: sequences to complete (mask of %REQ_FSEQ_*, can be zero)
* @error: whether an error occurred
*
* completion and trigger the next step.
*
* CONTEXT:
- * spin_lock_irq(q->queue_lock or q->mq_flush_lock)
+ * spin_lock_irq(q->queue_lock or fq->mq_flush_lock)
*
* RETURNS:
* %true if requests were added to the dispatch queue, %false otherwise.
*/
-static bool blk_flush_complete_seq(struct request *rq, unsigned int seq,
- int error)
+static bool blk_flush_complete_seq(struct request *rq,
+ struct blk_flush_queue *fq,
+ unsigned int seq, int error)
{
struct request_queue *q = rq->q;
- struct list_head *pending = &q->flush_queue[q->flush_pending_idx];
+ struct list_head *pending = &fq->flush_queue[fq->flush_pending_idx];
bool queued = false, kicked;
BUG_ON(rq->flush.seq & seq);
case REQ_FSEQ_POSTFLUSH:
/* queue for flush */
if (list_empty(pending))
- q->flush_pending_since = jiffies;
+ fq->flush_pending_since = jiffies;
list_move_tail(&rq->flush.list, pending);
break;
case REQ_FSEQ_DATA:
- list_move_tail(&rq->flush.list, &q->flush_data_in_flight);
+ list_move_tail(&rq->flush.list, &fq->flush_data_in_flight);
queued = blk_flush_queue_rq(rq, true);
break;
list_del_init(&rq->flush.list);
blk_flush_restore_request(rq);
if (q->mq_ops)
- blk_mq_end_io(rq, error);
+ blk_mq_end_request(rq, error);
else
__blk_end_request_all(rq, error);
break;
BUG();
}
- kicked = blk_kick_flush(q);
+ kicked = blk_kick_flush(q, fq);
return kicked | queued;
}
bool queued = false;
struct request *rq, *n;
unsigned long flags = 0;
+ struct blk_flush_queue *fq = blk_get_flush_queue(q, flush_rq->mq_ctx);
if (q->mq_ops) {
- spin_lock_irqsave(&q->mq_flush_lock, flags);
- q->flush_rq->tag = -1;
+ spin_lock_irqsave(&fq->mq_flush_lock, flags);
+ flush_rq->tag = -1;
}
- running = &q->flush_queue[q->flush_running_idx];
- BUG_ON(q->flush_pending_idx == q->flush_running_idx);
+ running = &fq->flush_queue[fq->flush_running_idx];
+ BUG_ON(fq->flush_pending_idx == fq->flush_running_idx);
/* account completion of the flush request */
- q->flush_running_idx ^= 1;
+ fq->flush_running_idx ^= 1;
if (!q->mq_ops)
elv_completed_request(q, flush_rq);
unsigned int seq = blk_flush_cur_seq(rq);
BUG_ON(seq != REQ_FSEQ_PREFLUSH && seq != REQ_FSEQ_POSTFLUSH);
- queued |= blk_flush_complete_seq(rq, seq, error);
+ queued |= blk_flush_complete_seq(rq, fq, seq, error);
}
/*
* directly into request_fn may confuse the driver. Always use
* kblockd.
*/
- if (queued || q->flush_queue_delayed) {
+ if (queued || fq->flush_queue_delayed) {
WARN_ON(q->mq_ops);
blk_run_queue_async(q);
}
- q->flush_queue_delayed = 0;
+ fq->flush_queue_delayed = 0;
if (q->mq_ops)
- spin_unlock_irqrestore(&q->mq_flush_lock, flags);
+ spin_unlock_irqrestore(&fq->mq_flush_lock, flags);
}
/**
* blk_kick_flush - consider issuing flush request
* @q: request_queue being kicked
+ * @fq: flush queue
*
* Flush related states of @q have changed, consider issuing flush request.
* Please read the comment at the top of this file for more info.
*
* CONTEXT:
- * spin_lock_irq(q->queue_lock or q->mq_flush_lock)
+ * spin_lock_irq(q->queue_lock or fq->mq_flush_lock)
*
* RETURNS:
* %true if flush was issued, %false otherwise.
*/
-static bool blk_kick_flush(struct request_queue *q)
+static bool blk_kick_flush(struct request_queue *q, struct blk_flush_queue *fq)
{
- struct list_head *pending = &q->flush_queue[q->flush_pending_idx];
+ struct list_head *pending = &fq->flush_queue[fq->flush_pending_idx];
struct request *first_rq =
list_first_entry(pending, struct request, flush.list);
+ struct request *flush_rq = fq->flush_rq;
/* C1 described at the top of this file */
- if (q->flush_pending_idx != q->flush_running_idx || list_empty(pending))
+ if (fq->flush_pending_idx != fq->flush_running_idx || list_empty(pending))
return false;
/* C2 and C3 */
- if (!list_empty(&q->flush_data_in_flight) &&
+ if (!list_empty(&fq->flush_data_in_flight) &&
time_before(jiffies,
- q->flush_pending_since + FLUSH_PENDING_TIMEOUT))
+ fq->flush_pending_since + FLUSH_PENDING_TIMEOUT))
return false;
/*
* Issue flush and toggle pending_idx. This makes pending_idx
* different from running_idx, which means flush is in flight.
*/
- q->flush_pending_idx ^= 1;
+ fq->flush_pending_idx ^= 1;
- blk_rq_init(q, q->flush_rq);
- if (q->mq_ops)
- blk_mq_clone_flush_request(q->flush_rq, first_rq);
+ blk_rq_init(q, flush_rq);
+
+ /*
+ * Borrow tag from the first request since they can't
+ * be in flight at the same time.
+ */
+ if (q->mq_ops) {
+ flush_rq->mq_ctx = first_rq->mq_ctx;
+ flush_rq->tag = first_rq->tag;
+ }
- q->flush_rq->cmd_type = REQ_TYPE_FS;
- q->flush_rq->cmd_flags = WRITE_FLUSH | REQ_FLUSH_SEQ;
- q->flush_rq->rq_disk = first_rq->rq_disk;
- q->flush_rq->end_io = flush_end_io;
+ flush_rq->cmd_type = REQ_TYPE_FS;
+ flush_rq->cmd_flags = WRITE_FLUSH | REQ_FLUSH_SEQ;
+ flush_rq->rq_disk = first_rq->rq_disk;
+ flush_rq->end_io = flush_end_io;
- return blk_flush_queue_rq(q->flush_rq, false);
+ return blk_flush_queue_rq(flush_rq, false);
}
static void flush_data_end_io(struct request *rq, int error)
{
struct request_queue *q = rq->q;
+ struct blk_flush_queue *fq = blk_get_flush_queue(q, NULL);
/*
* After populating an empty queue, kick it to avoid stall. Read
* the comment in flush_end_io().
*/
- if (blk_flush_complete_seq(rq, REQ_FSEQ_DATA, error))
+ if (blk_flush_complete_seq(rq, fq, REQ_FSEQ_DATA, error))
blk_run_queue_async(q);
}
{
struct request_queue *q = rq->q;
struct blk_mq_hw_ctx *hctx;
- struct blk_mq_ctx *ctx;
+ struct blk_mq_ctx *ctx = rq->mq_ctx;
unsigned long flags;
+ struct blk_flush_queue *fq = blk_get_flush_queue(q, ctx);
- ctx = rq->mq_ctx;
hctx = q->mq_ops->map_queue(q, ctx->cpu);
/*
* After populating an empty queue, kick it to avoid stall. Read
* the comment in flush_end_io().
*/
- spin_lock_irqsave(&q->mq_flush_lock, flags);
- if (blk_flush_complete_seq(rq, REQ_FSEQ_DATA, error))
+ spin_lock_irqsave(&fq->mq_flush_lock, flags);
+ if (blk_flush_complete_seq(rq, fq, REQ_FSEQ_DATA, error))
blk_mq_run_hw_queue(hctx, true);
- spin_unlock_irqrestore(&q->mq_flush_lock, flags);
+ spin_unlock_irqrestore(&fq->mq_flush_lock, flags);
}
/**
struct request_queue *q = rq->q;
unsigned int fflags = q->flush_flags; /* may change, cache */
unsigned int policy = blk_flush_policy(fflags, rq);
+ struct blk_flush_queue *fq = blk_get_flush_queue(q, rq->mq_ctx);
/*
* @policy now records what operations need to be done. Adjust
*/
if (!policy) {
if (q->mq_ops)
- blk_mq_end_io(rq, 0);
+ blk_mq_end_request(rq, 0);
else
__blk_end_bidi_request(rq, 0, 0, 0);
return;
if (q->mq_ops) {
rq->end_io = mq_flush_data_end_io;
- spin_lock_irq(&q->mq_flush_lock);
- blk_flush_complete_seq(rq, REQ_FSEQ_ACTIONS & ~policy, 0);
- spin_unlock_irq(&q->mq_flush_lock);
+ spin_lock_irq(&fq->mq_flush_lock);
+ blk_flush_complete_seq(rq, fq, REQ_FSEQ_ACTIONS & ~policy, 0);
+ spin_unlock_irq(&fq->mq_flush_lock);
return;
}
rq->end_io = flush_data_end_io;
- blk_flush_complete_seq(rq, REQ_FSEQ_ACTIONS & ~policy, 0);
+ blk_flush_complete_seq(rq, fq, REQ_FSEQ_ACTIONS & ~policy, 0);
}
/**
}
EXPORT_SYMBOL(blkdev_issue_flush);
-void blk_mq_init_flush(struct request_queue *q)
+struct blk_flush_queue *blk_alloc_flush_queue(struct request_queue *q,
+ int node, int cmd_size)
+{
+ struct blk_flush_queue *fq;
+ int rq_sz = sizeof(struct request);
+
+ fq = kzalloc_node(sizeof(*fq), GFP_KERNEL, node);
+ if (!fq)
+ goto fail;
+
+ if (q->mq_ops) {
+ spin_lock_init(&fq->mq_flush_lock);
+ rq_sz = round_up(rq_sz + cmd_size, cache_line_size());
+ }
+
+ fq->flush_rq = kzalloc_node(rq_sz, GFP_KERNEL, node);
+ if (!fq->flush_rq)
+ goto fail_rq;
+
+ INIT_LIST_HEAD(&fq->flush_queue[0]);
+ INIT_LIST_HEAD(&fq->flush_queue[1]);
+ INIT_LIST_HEAD(&fq->flush_data_in_flight);
+
+ return fq;
+
+ fail_rq:
+ kfree(fq);
+ fail:
+ return NULL;
+}
+
+void blk_free_flush_queue(struct blk_flush_queue *fq)
{
- spin_lock_init(&q->mq_flush_lock);
+ /* bio based request queue hasn't flush queue */
+ if (!fq)
+ return;
+
+ kfree(fq->flush_rq);
+ kfree(fq);
}
if (!b1 || !b2)
return -1;
- if (b1->sector_size != b2->sector_size) {
- printk(KERN_ERR "%s: %s/%s sector sz %u != %u\n", __func__,
- gd1->disk_name, gd2->disk_name,
- b1->sector_size, b2->sector_size);
+ if (b1->interval != b2->interval) {
+ pr_err("%s: %s/%s protection interval %u != %u\n",
+ __func__, gd1->disk_name, gd2->disk_name,
+ b1->interval, b2->interval);
return -1;
}
}
EXPORT_SYMBOL(blk_integrity_compare);
-int blk_integrity_merge_rq(struct request_queue *q, struct request *req,
- struct request *next)
+bool blk_integrity_merge_rq(struct request_queue *q, struct request *req,
+ struct request *next)
{
- if (blk_integrity_rq(req) != blk_integrity_rq(next))
- return -1;
+ if (blk_integrity_rq(req) == 0 && blk_integrity_rq(next) == 0)
+ return true;
+
+ if (blk_integrity_rq(req) == 0 || blk_integrity_rq(next) == 0)
+ return false;
+
+ if (bio_integrity(req->bio)->bip_flags !=
+ bio_integrity(next->bio)->bip_flags)
+ return false;
if (req->nr_integrity_segments + next->nr_integrity_segments >
q->limits.max_integrity_segments)
- return -1;
+ return false;
- return 0;
+ return true;
}
EXPORT_SYMBOL(blk_integrity_merge_rq);
-int blk_integrity_merge_bio(struct request_queue *q, struct request *req,
- struct bio *bio)
+bool blk_integrity_merge_bio(struct request_queue *q, struct request *req,
+ struct bio *bio)
{
int nr_integrity_segs;
struct bio *next = bio->bi_next;
+ if (blk_integrity_rq(req) == 0 && bio_integrity(bio) == NULL)
+ return true;
+
+ if (blk_integrity_rq(req) == 0 || bio_integrity(bio) == NULL)
+ return false;
+
+ if (bio_integrity(req->bio)->bip_flags != bio_integrity(bio)->bip_flags)
+ return false;
+
bio->bi_next = NULL;
nr_integrity_segs = blk_rq_count_integrity_sg(q, bio);
bio->bi_next = next;
if (req->nr_integrity_segments + nr_integrity_segs >
q->limits.max_integrity_segments)
- return -1;
+ return false;
req->nr_integrity_segments += nr_integrity_segs;
- return 0;
+ return true;
}
EXPORT_SYMBOL(blk_integrity_merge_bio);
return sprintf(page, "0\n");
}
-static ssize_t integrity_read_store(struct blk_integrity *bi,
- const char *page, size_t count)
+static ssize_t integrity_verify_store(struct blk_integrity *bi,
+ const char *page, size_t count)
{
char *p = (char *) page;
unsigned long val = simple_strtoul(p, &p, 10);
if (val)
- bi->flags |= INTEGRITY_FLAG_READ;
+ bi->flags |= BLK_INTEGRITY_VERIFY;
else
- bi->flags &= ~INTEGRITY_FLAG_READ;
+ bi->flags &= ~BLK_INTEGRITY_VERIFY;
return count;
}
-static ssize_t integrity_read_show(struct blk_integrity *bi, char *page)
+static ssize_t integrity_verify_show(struct blk_integrity *bi, char *page)
{
- return sprintf(page, "%d\n", (bi->flags & INTEGRITY_FLAG_READ) != 0);
+ return sprintf(page, "%d\n", (bi->flags & BLK_INTEGRITY_VERIFY) != 0);
}
-static ssize_t integrity_write_store(struct blk_integrity *bi,
- const char *page, size_t count)
+static ssize_t integrity_generate_store(struct blk_integrity *bi,
+ const char *page, size_t count)
{
char *p = (char *) page;
unsigned long val = simple_strtoul(p, &p, 10);
if (val)
- bi->flags |= INTEGRITY_FLAG_WRITE;
+ bi->flags |= BLK_INTEGRITY_GENERATE;
else
- bi->flags &= ~INTEGRITY_FLAG_WRITE;
+ bi->flags &= ~BLK_INTEGRITY_GENERATE;
return count;
}
-static ssize_t integrity_write_show(struct blk_integrity *bi, char *page)
+static ssize_t integrity_generate_show(struct blk_integrity *bi, char *page)
+{
+ return sprintf(page, "%d\n", (bi->flags & BLK_INTEGRITY_GENERATE) != 0);
+}
+
+static ssize_t integrity_device_show(struct blk_integrity *bi, char *page)
{
- return sprintf(page, "%d\n", (bi->flags & INTEGRITY_FLAG_WRITE) != 0);
+ return sprintf(page, "%u\n",
+ (bi->flags & BLK_INTEGRITY_DEVICE_CAPABLE) != 0);
}
static struct integrity_sysfs_entry integrity_format_entry = {
.show = integrity_tag_size_show,
};
-static struct integrity_sysfs_entry integrity_read_entry = {
+static struct integrity_sysfs_entry integrity_verify_entry = {
.attr = { .name = "read_verify", .mode = S_IRUGO | S_IWUSR },
- .show = integrity_read_show,
- .store = integrity_read_store,
+ .show = integrity_verify_show,
+ .store = integrity_verify_store,
};
-static struct integrity_sysfs_entry integrity_write_entry = {
+static struct integrity_sysfs_entry integrity_generate_entry = {
.attr = { .name = "write_generate", .mode = S_IRUGO | S_IWUSR },
- .show = integrity_write_show,
- .store = integrity_write_store,
+ .show = integrity_generate_show,
+ .store = integrity_generate_store,
+};
+
+static struct integrity_sysfs_entry integrity_device_entry = {
+ .attr = { .name = "device_is_integrity_capable", .mode = S_IRUGO },
+ .show = integrity_device_show,
};
static struct attribute *integrity_attrs[] = {
&integrity_format_entry.attr,
&integrity_tag_size_entry.attr,
- &integrity_read_entry.attr,
- &integrity_write_entry.attr,
+ &integrity_verify_entry.attr,
+ &integrity_generate_entry.attr,
+ &integrity_device_entry.attr,
NULL,
};
kobject_uevent(&bi->kobj, KOBJ_ADD);
- bi->flags |= INTEGRITY_FLAG_READ | INTEGRITY_FLAG_WRITE;
- bi->sector_size = queue_logical_block_size(disk->queue);
+ bi->flags |= BLK_INTEGRITY_VERIFY | BLK_INTEGRITY_GENERATE;
+ bi->interval = queue_logical_block_size(disk->queue);
disk->integrity = bi;
} else
bi = disk->integrity;
bi->generate_fn = template->generate_fn;
bi->verify_fn = template->verify_fn;
bi->tuple_size = template->tuple_size;
- bi->set_tag_fn = template->set_tag_fn;
- bi->get_tag_fn = template->get_tag_fn;
bi->tag_size = template->tag_size;
+ bi->flags |= template->flags;
} else
bi->name = bi_unsupported_name;
void blk_recount_segments(struct request_queue *q, struct bio *bio)
{
- if (test_bit(QUEUE_FLAG_NO_SG_MERGE, &q->queue_flags) &&
+ bool no_sg_merge = !!test_bit(QUEUE_FLAG_NO_SG_MERGE,
+ &q->queue_flags);
+
+ if (no_sg_merge && !bio_flagged(bio, BIO_CLONED) &&
bio->bi_vcnt < queue_max_segments(q))
bio->bi_phys_segments = bio->bi_vcnt;
else {
struct bio *nxt = bio->bi_next;
bio->bi_next = NULL;
- bio->bi_phys_segments = __blk_recalc_rq_segments(q, bio, false);
+ bio->bi_phys_segments = __blk_recalc_rq_segments(q, bio,
+ no_sg_merge);
bio->bi_next = nxt;
}
if (req->nr_phys_segments + nr_phys_segs > queue_max_segments(q))
goto no_merge;
- if (bio_integrity(bio) && blk_integrity_merge_bio(q, req, bio))
+ if (blk_integrity_merge_bio(q, req, bio) == false)
goto no_merge;
/*
if (total_phys_segments > queue_max_segments(q))
return 0;
- if (blk_integrity_rq(req) && blk_integrity_merge_rq(q, req, next))
+ if (blk_integrity_merge_rq(q, req, next) == false)
return 0;
/* Merge is OK... */
return false;
/* only merge integrity protected bio into ditto rq */
- if (bio_integrity(bio) != blk_integrity_rq(rq))
+ if (blk_integrity_merge_bio(rq->q, rq, bio) == false)
return false;
/* must be using the same buffer */
return;
wait_cnt = atomic_dec_return(&bs->wait_cnt);
+ if (unlikely(wait_cnt < 0))
+ wait_cnt = atomic_inc_return(&bs->wait_cnt);
if (wait_cnt == 0) {
-wake:
atomic_add(bt->wake_cnt, &bs->wait_cnt);
bt_index_atomic_inc(&bt->wake_index);
wake_up(&bs->wait);
- } else if (wait_cnt < 0) {
- wait_cnt = atomic_inc_return(&bs->wait_cnt);
- if (!wait_cnt)
- goto wake;
}
}
__blk_mq_put_reserved_tag(tags, tag);
}
-static void bt_for_each_free(struct blk_mq_bitmap_tags *bt,
- unsigned long *free_map, unsigned int off)
+static void bt_for_each(struct blk_mq_hw_ctx *hctx,
+ struct blk_mq_bitmap_tags *bt, unsigned int off,
+ busy_iter_fn *fn, void *data, bool reserved)
{
- int i;
+ struct request *rq;
+ int bit, i;
for (i = 0; i < bt->map_nr; i++) {
struct blk_align_bitmap *bm = &bt->map[i];
- int bit = 0;
- do {
- bit = find_next_zero_bit(&bm->word, bm->depth, bit);
- if (bit >= bm->depth)
- break;
-
- __set_bit(bit + off, free_map);
- bit++;
- } while (1);
+ for (bit = find_first_bit(&bm->word, bm->depth);
+ bit < bm->depth;
+ bit = find_next_bit(&bm->word, bm->depth, bit + 1)) {
+ rq = blk_mq_tag_to_rq(hctx->tags, off + bit);
+ if (rq->q == hctx->queue)
+ fn(hctx, rq, data, reserved);
+ }
off += (1 << bt->bits_per_word);
}
}
-void blk_mq_tag_busy_iter(struct blk_mq_tags *tags,
- void (*fn)(void *, unsigned long *), void *data)
+void blk_mq_tag_busy_iter(struct blk_mq_hw_ctx *hctx, busy_iter_fn *fn,
+ void *priv)
{
- unsigned long *tag_map;
- size_t map_size;
-
- map_size = ALIGN(tags->nr_tags, BITS_PER_LONG) / BITS_PER_LONG;
- tag_map = kzalloc(map_size * sizeof(unsigned long), GFP_ATOMIC);
- if (!tag_map)
- return;
+ struct blk_mq_tags *tags = hctx->tags;
- bt_for_each_free(&tags->bitmap_tags, tag_map, tags->nr_reserved_tags);
if (tags->nr_reserved_tags)
- bt_for_each_free(&tags->breserved_tags, tag_map, 0);
-
- fn(data, tag_map);
- kfree(tag_map);
+ bt_for_each(hctx, &tags->breserved_tags, 0, fn, priv, true);
+ bt_for_each(hctx, &tags->bitmap_tags, tags->nr_reserved_tags, fn, priv,
+ false);
}
EXPORT_SYMBOL(blk_mq_tag_busy_iter);
}
bt->wake_cnt = BT_WAIT_BATCH;
- if (bt->wake_cnt > depth / 4)
- bt->wake_cnt = max(1U, depth / 4);
+ if (bt->wake_cnt > depth / BT_WAIT_QUEUES)
+ bt->wake_cnt = max(1U, depth / BT_WAIT_QUEUES);
bt->depth = depth;
}
#include <linux/cache.h>
#include <linux/sched/sysctl.h>
#include <linux/delay.h>
+#include <linux/crash_dump.h>
#include <trace/events/block.h>
struct blk_mq_hw_ctx *hctx;
struct request *rq;
struct blk_mq_alloc_data alloc_data;
+ int ret;
- if (blk_mq_queue_enter(q))
- return NULL;
+ ret = blk_mq_queue_enter(q);
+ if (ret)
+ return ERR_PTR(ret);
ctx = blk_mq_get_ctx(q);
hctx = q->mq_ops->map_queue(q, ctx->cpu);
ctx = alloc_data.ctx;
}
blk_mq_put_ctx(ctx);
+ if (!rq)
+ return ERR_PTR(-EWOULDBLOCK);
return rq;
}
EXPORT_SYMBOL(blk_mq_alloc_request);
__blk_mq_free_request(hctx, ctx, rq);
}
-/*
- * Clone all relevant state from a request that has been put on hold in
- * the flush state machine into the preallocated flush request that hangs
- * off the request queue.
- *
- * For a driver the flush request should be invisible, that's why we are
- * impersonating the original request here.
- */
-void blk_mq_clone_flush_request(struct request *flush_rq,
- struct request *orig_rq)
-{
- struct blk_mq_hw_ctx *hctx =
- orig_rq->q->mq_ops->map_queue(orig_rq->q, orig_rq->mq_ctx->cpu);
-
- flush_rq->mq_ctx = orig_rq->mq_ctx;
- flush_rq->tag = orig_rq->tag;
- memcpy(blk_mq_rq_to_pdu(flush_rq), blk_mq_rq_to_pdu(orig_rq),
- hctx->cmd_size);
-}
-
-inline void __blk_mq_end_io(struct request *rq, int error)
+inline void __blk_mq_end_request(struct request *rq, int error)
{
blk_account_io_done(rq);
blk_mq_free_request(rq);
}
}
-EXPORT_SYMBOL(__blk_mq_end_io);
+EXPORT_SYMBOL(__blk_mq_end_request);
-void blk_mq_end_io(struct request *rq, int error)
+void blk_mq_end_request(struct request *rq, int error)
{
if (blk_update_request(rq, error, blk_rq_bytes(rq)))
BUG();
- __blk_mq_end_io(rq, error);
+ __blk_mq_end_request(rq, error);
}
-EXPORT_SYMBOL(blk_mq_end_io);
+EXPORT_SYMBOL(blk_mq_end_request);
static void __blk_mq_complete_request_remote(void *data)
{
struct request_queue *q = rq->q;
if (!q->softirq_done_fn)
- blk_mq_end_io(rq, rq->errors);
+ blk_mq_end_request(rq, rq->errors);
else
blk_mq_ipi_complete_request(rq);
}
}
EXPORT_SYMBOL(blk_mq_complete_request);
-static void blk_mq_start_request(struct request *rq, bool last)
+void blk_mq_start_request(struct request *rq)
{
struct request_queue *q = rq->q;
*/
rq->nr_phys_segments++;
}
-
- /*
- * Flag the last request in the series so that drivers know when IO
- * should be kicked off, if they don't do it on a per-request basis.
- *
- * Note: the flag isn't the only condition drivers should do kick off.
- * If drive is busy, the last request might not have the bit set.
- */
- if (last)
- rq->cmd_flags |= REQ_END;
}
+EXPORT_SYMBOL(blk_mq_start_request);
static void __blk_mq_requeue_request(struct request *rq)
{
struct request_queue *q = rq->q;
trace_block_rq_requeue(q, rq);
- clear_bit(REQ_ATOM_STARTED, &rq->atomic_flags);
-
- rq->cmd_flags &= ~REQ_END;
- if (q->dma_drain_size && blk_rq_bytes(rq))
- rq->nr_phys_segments--;
+ if (test_and_clear_bit(REQ_ATOM_STARTED, &rq->atomic_flags)) {
+ if (q->dma_drain_size && blk_rq_bytes(rq))
+ rq->nr_phys_segments--;
+ }
}
void blk_mq_requeue_request(struct request *rq)
{
__blk_mq_requeue_request(rq);
- blk_clear_rq_complete(rq);
BUG_ON(blk_queued_rq(rq));
blk_mq_add_to_requeue_list(rq, true);
}
EXPORT_SYMBOL(blk_mq_kick_requeue_list);
-static inline bool is_flush_request(struct request *rq, unsigned int tag)
+static inline bool is_flush_request(struct request *rq,
+ struct blk_flush_queue *fq, unsigned int tag)
{
return ((rq->cmd_flags & REQ_FLUSH_SEQ) &&
- rq->q->flush_rq->tag == tag);
+ fq->flush_rq->tag == tag);
}
struct request *blk_mq_tag_to_rq(struct blk_mq_tags *tags, unsigned int tag)
{
struct request *rq = tags->rqs[tag];
+ /* mq_ctx of flush rq is always cloned from the corresponding req */
+ struct blk_flush_queue *fq = blk_get_flush_queue(rq->q, rq->mq_ctx);
- if (!is_flush_request(rq, tag))
+ if (!is_flush_request(rq, fq, tag))
return rq;
- return rq->q->flush_rq;
+ return fq->flush_rq;
}
EXPORT_SYMBOL(blk_mq_tag_to_rq);
struct blk_mq_timeout_data {
- struct blk_mq_hw_ctx *hctx;
- unsigned long *next;
- unsigned int *next_set;
+ unsigned long next;
+ unsigned int next_set;
};
-static void blk_mq_timeout_check(void *__data, unsigned long *free_tags)
+void blk_mq_rq_timed_out(struct request *req, bool reserved)
{
- struct blk_mq_timeout_data *data = __data;
- struct blk_mq_hw_ctx *hctx = data->hctx;
- unsigned int tag;
-
- /* It may not be in flight yet (this is where
- * the REQ_ATOMIC_STARTED flag comes in). The requests are
- * statically allocated, so we know it's always safe to access the
- * memory associated with a bit offset into ->rqs[].
- */
- tag = 0;
- do {
- struct request *rq;
-
- tag = find_next_zero_bit(free_tags, hctx->tags->nr_tags, tag);
- if (tag >= hctx->tags->nr_tags)
- break;
-
- rq = blk_mq_tag_to_rq(hctx->tags, tag++);
- if (rq->q != hctx->queue)
- continue;
- if (!test_bit(REQ_ATOM_STARTED, &rq->atomic_flags))
- continue;
-
- blk_rq_check_expired(rq, data->next, data->next_set);
- } while (1);
-}
-
-static void blk_mq_hw_ctx_check_timeout(struct blk_mq_hw_ctx *hctx,
- unsigned long *next,
- unsigned int *next_set)
-{
- struct blk_mq_timeout_data data = {
- .hctx = hctx,
- .next = next,
- .next_set = next_set,
- };
-
- /*
- * Ask the tagging code to iterate busy requests, so we can
- * check them for timeout.
- */
- blk_mq_tag_busy_iter(hctx->tags, blk_mq_timeout_check, &data);
-}
-
-static enum blk_eh_timer_return blk_mq_rq_timed_out(struct request *rq)
-{
- struct request_queue *q = rq->q;
+ struct blk_mq_ops *ops = req->q->mq_ops;
+ enum blk_eh_timer_return ret = BLK_EH_RESET_TIMER;
/*
* We know that complete is set at this point. If STARTED isn't set
* we both flags will get cleared. So check here again, and ignore
* a timeout event with a request that isn't active.
*/
- if (!test_bit(REQ_ATOM_STARTED, &rq->atomic_flags))
- return BLK_EH_NOT_HANDLED;
+ if (!test_bit(REQ_ATOM_STARTED, &req->atomic_flags))
+ return;
+
+ if (ops->timeout)
+ ret = ops->timeout(req, reserved);
+
+ switch (ret) {
+ case BLK_EH_HANDLED:
+ __blk_mq_complete_request(req);
+ break;
+ case BLK_EH_RESET_TIMER:
+ blk_add_timer(req);
+ blk_clear_rq_complete(req);
+ break;
+ case BLK_EH_NOT_HANDLED:
+ break;
+ default:
+ printk(KERN_ERR "block: bad eh return: %d\n", ret);
+ break;
+ }
+}
+
+static void blk_mq_check_expired(struct blk_mq_hw_ctx *hctx,
+ struct request *rq, void *priv, bool reserved)
+{
+ struct blk_mq_timeout_data *data = priv;
- if (!q->mq_ops->timeout)
- return BLK_EH_RESET_TIMER;
+ if (!test_bit(REQ_ATOM_STARTED, &rq->atomic_flags))
+ return;
- return q->mq_ops->timeout(rq);
+ if (time_after_eq(jiffies, rq->deadline)) {
+ if (!blk_mark_rq_complete(rq))
+ blk_mq_rq_timed_out(rq, reserved);
+ } else if (!data->next_set || time_after(data->next, rq->deadline)) {
+ data->next = rq->deadline;
+ data->next_set = 1;
+ }
}
-static void blk_mq_rq_timer(unsigned long data)
+static void blk_mq_rq_timer(unsigned long priv)
{
- struct request_queue *q = (struct request_queue *) data;
+ struct request_queue *q = (struct request_queue *)priv;
+ struct blk_mq_timeout_data data = {
+ .next = 0,
+ .next_set = 0,
+ };
struct blk_mq_hw_ctx *hctx;
- unsigned long next = 0;
- int i, next_set = 0;
+ int i;
queue_for_each_hw_ctx(q, hctx, i) {
/*
if (!hctx->nr_ctx || !hctx->tags)
continue;
- blk_mq_hw_ctx_check_timeout(hctx, &next, &next_set);
+ blk_mq_tag_busy_iter(hctx, blk_mq_check_expired, &data);
}
- if (next_set) {
- next = blk_rq_timeout(round_jiffies_up(next));
- mod_timer(&q->timeout, next);
+ if (data.next_set) {
+ data.next = blk_rq_timeout(round_jiffies_up(data.next));
+ mod_timer(&q->timeout, data.next);
} else {
queue_for_each_hw_ctx(q, hctx, i)
blk_mq_tag_idle(hctx);
rq = list_first_entry(&rq_list, struct request, queuelist);
list_del_init(&rq->queuelist);
- blk_mq_start_request(rq, list_empty(&rq_list));
-
- ret = q->mq_ops->queue_rq(hctx, rq);
+ ret = q->mq_ops->queue_rq(hctx, rq, list_empty(&rq_list));
switch (ret) {
case BLK_MQ_RQ_QUEUE_OK:
queued++;
pr_err("blk-mq: bad return on queue: %d\n", ret);
case BLK_MQ_RQ_QUEUE_ERROR:
rq->errors = -EIO;
- blk_mq_end_io(rq, rq->errors);
+ blk_mq_end_request(rq, rq->errors);
break;
}
int ret;
blk_mq_bio_to_request(rq, bio);
- blk_mq_start_request(rq, true);
/*
* For OK queue, we are done. For error, kill it. Any other
* error (busy), just add it to our list as we previously
* would have done
*/
- ret = q->mq_ops->queue_rq(data.hctx, rq);
+ ret = q->mq_ops->queue_rq(data.hctx, rq, true);
if (ret == BLK_MQ_RQ_QUEUE_OK)
goto done;
else {
if (ret == BLK_MQ_RQ_QUEUE_ERROR) {
rq->errors = -EIO;
- blk_mq_end_io(rq, rq->errors);
+ blk_mq_end_request(rq, rq->errors);
goto done;
}
}
return NOTIFY_OK;
}
+static void blk_mq_exit_hctx(struct request_queue *q,
+ struct blk_mq_tag_set *set,
+ struct blk_mq_hw_ctx *hctx, unsigned int hctx_idx)
+{
+ unsigned flush_start_tag = set->queue_depth;
+
+ blk_mq_tag_idle(hctx);
+
+ if (set->ops->exit_request)
+ set->ops->exit_request(set->driver_data,
+ hctx->fq->flush_rq, hctx_idx,
+ flush_start_tag + hctx_idx);
+
+ if (set->ops->exit_hctx)
+ set->ops->exit_hctx(hctx, hctx_idx);
+
+ blk_mq_unregister_cpu_notifier(&hctx->cpu_notifier);
+ blk_free_flush_queue(hctx->fq);
+ kfree(hctx->ctxs);
+ blk_mq_free_bitmap(&hctx->ctx_map);
+}
+
static void blk_mq_exit_hw_queues(struct request_queue *q,
struct blk_mq_tag_set *set, int nr_queue)
{
queue_for_each_hw_ctx(q, hctx, i) {
if (i == nr_queue)
break;
-
- blk_mq_tag_idle(hctx);
-
- if (set->ops->exit_hctx)
- set->ops->exit_hctx(hctx, i);
-
- blk_mq_unregister_cpu_notifier(&hctx->cpu_notifier);
- kfree(hctx->ctxs);
- blk_mq_free_bitmap(&hctx->ctx_map);
+ blk_mq_exit_hctx(q, set, hctx, i);
}
-
}
static void blk_mq_free_hw_queues(struct request_queue *q,
}
}
-static int blk_mq_init_hw_queues(struct request_queue *q,
- struct blk_mq_tag_set *set)
+static int blk_mq_init_hctx(struct request_queue *q,
+ struct blk_mq_tag_set *set,
+ struct blk_mq_hw_ctx *hctx, unsigned hctx_idx)
{
- struct blk_mq_hw_ctx *hctx;
- unsigned int i;
+ int node;
+ unsigned flush_start_tag = set->queue_depth;
+
+ node = hctx->numa_node;
+ if (node == NUMA_NO_NODE)
+ node = hctx->numa_node = set->numa_node;
+
+ INIT_DELAYED_WORK(&hctx->run_work, blk_mq_run_work_fn);
+ INIT_DELAYED_WORK(&hctx->delay_work, blk_mq_delay_work_fn);
+ spin_lock_init(&hctx->lock);
+ INIT_LIST_HEAD(&hctx->dispatch);
+ hctx->queue = q;
+ hctx->queue_num = hctx_idx;
+ hctx->flags = set->flags;
+ hctx->cmd_size = set->cmd_size;
+
+ blk_mq_init_cpu_notifier(&hctx->cpu_notifier,
+ blk_mq_hctx_notify, hctx);
+ blk_mq_register_cpu_notifier(&hctx->cpu_notifier);
+
+ hctx->tags = set->tags[hctx_idx];
/*
- * Initialize hardware queues
+ * Allocate space for all possible cpus to avoid allocation at
+ * runtime
*/
- queue_for_each_hw_ctx(q, hctx, i) {
- int node;
+ hctx->ctxs = kmalloc_node(nr_cpu_ids * sizeof(void *),
+ GFP_KERNEL, node);
+ if (!hctx->ctxs)
+ goto unregister_cpu_notifier;
- node = hctx->numa_node;
- if (node == NUMA_NO_NODE)
- node = hctx->numa_node = set->numa_node;
+ if (blk_mq_alloc_bitmap(&hctx->ctx_map, node))
+ goto free_ctxs;
- INIT_DELAYED_WORK(&hctx->run_work, blk_mq_run_work_fn);
- INIT_DELAYED_WORK(&hctx->delay_work, blk_mq_delay_work_fn);
- spin_lock_init(&hctx->lock);
- INIT_LIST_HEAD(&hctx->dispatch);
- hctx->queue = q;
- hctx->queue_num = i;
- hctx->flags = set->flags;
- hctx->cmd_size = set->cmd_size;
+ hctx->nr_ctx = 0;
- blk_mq_init_cpu_notifier(&hctx->cpu_notifier,
- blk_mq_hctx_notify, hctx);
- blk_mq_register_cpu_notifier(&hctx->cpu_notifier);
+ if (set->ops->init_hctx &&
+ set->ops->init_hctx(hctx, set->driver_data, hctx_idx))
+ goto free_bitmap;
- hctx->tags = set->tags[i];
+ hctx->fq = blk_alloc_flush_queue(q, hctx->numa_node, set->cmd_size);
+ if (!hctx->fq)
+ goto exit_hctx;
- /*
- * Allocate space for all possible cpus to avoid allocation at
- * runtime
- */
- hctx->ctxs = kmalloc_node(nr_cpu_ids * sizeof(void *),
- GFP_KERNEL, node);
- if (!hctx->ctxs)
- break;
+ if (set->ops->init_request &&
+ set->ops->init_request(set->driver_data,
+ hctx->fq->flush_rq, hctx_idx,
+ flush_start_tag + hctx_idx, node))
+ goto free_fq;
- if (blk_mq_alloc_bitmap(&hctx->ctx_map, node))
- break;
+ return 0;
- hctx->nr_ctx = 0;
+ free_fq:
+ kfree(hctx->fq);
+ exit_hctx:
+ if (set->ops->exit_hctx)
+ set->ops->exit_hctx(hctx, hctx_idx);
+ free_bitmap:
+ blk_mq_free_bitmap(&hctx->ctx_map);
+ free_ctxs:
+ kfree(hctx->ctxs);
+ unregister_cpu_notifier:
+ blk_mq_unregister_cpu_notifier(&hctx->cpu_notifier);
+
+ return -1;
+}
+
+static int blk_mq_init_hw_queues(struct request_queue *q,
+ struct blk_mq_tag_set *set)
+{
+ struct blk_mq_hw_ctx *hctx;
+ unsigned int i;
- if (set->ops->init_hctx &&
- set->ops->init_hctx(hctx, set->driver_data, i))
+ /*
+ * Initialize hardware queues
+ */
+ queue_for_each_hw_ctx(q, hctx, i) {
+ if (blk_mq_init_hctx(q, set, hctx, i))
break;
}
if (!ctx)
return ERR_PTR(-ENOMEM);
+ /*
+ * If a crashdump is active, then we are potentially in a very
+ * memory constrained environment. Limit us to 1 queue and
+ * 64 tags to prevent using too much memory.
+ */
+ if (is_kdump_kernel()) {
+ set->nr_hw_queues = 1;
+ set->queue_depth = min(64U, set->queue_depth);
+ }
+
hctxs = kmalloc_node(set->nr_hw_queues * sizeof(*hctxs), GFP_KERNEL,
set->numa_node);
if (!hctxs[i])
goto err_hctxs;
- if (!zalloc_cpumask_var(&hctxs[i]->cpumask, GFP_KERNEL))
+ if (!zalloc_cpumask_var_node(&hctxs[i]->cpumask, GFP_KERNEL,
+ node))
goto err_hctxs;
atomic_set(&hctxs[i]->nr_active, 0);
else
blk_queue_make_request(q, blk_sq_make_request);
- blk_queue_rq_timed_out(q, blk_mq_rq_timed_out);
if (set->timeout)
blk_queue_rq_timeout(q, set->timeout);
if (set->ops->complete)
blk_queue_softirq_done(q, set->ops->complete);
- blk_mq_init_flush(q);
blk_mq_init_cpu_queues(q, set->nr_hw_queues);
- q->flush_rq = kzalloc(round_up(sizeof(struct request) +
- set->cmd_size, cache_line_size()),
- GFP_KERNEL);
- if (!q->flush_rq)
- goto err_hw;
-
if (blk_mq_init_hw_queues(q, set))
- goto err_flush_rq;
+ goto err_hw;
mutex_lock(&all_q_mutex);
list_add_tail(&q->all_q_node, &all_q_list);
return q;
-err_flush_rq:
- kfree(q->flush_rq);
err_hw:
blk_cleanup_queue(q);
err_hctxs:
void __blk_mq_complete_request(struct request *rq);
void blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx, bool async);
-void blk_mq_init_flush(struct request_queue *q);
void blk_mq_freeze_queue(struct request_queue *q);
void blk_mq_free_queue(struct request_queue *q);
void blk_mq_clone_flush_request(struct request *flush_rq,
extern int blk_mq_sysfs_register(struct request_queue *q);
extern void blk_mq_sysfs_unregister(struct request_queue *q);
+extern void blk_mq_rq_timed_out(struct request *req, bool reserved);
+
/*
* Basic implementation of sparser bitmap, allowing the user to spread
* the bits over more cachelines.
bottom = max(b->physical_block_size, b->io_min) + alignment;
/* Verify that top and bottom intervals line up */
- if (max(top, bottom) & (min(top, bottom) - 1)) {
+ if (max(top, bottom) % min(top, bottom)) {
t->misaligned = 1;
ret = -1;
}
/* Find lowest common alignment_offset */
t->alignment_offset = lcm(t->alignment_offset, alignment)
- & (max(t->physical_block_size, t->io_min) - 1);
+ % max(t->physical_block_size, t->io_min);
/* Verify that new alignment_offset is on a logical block boundary */
if (t->alignment_offset & (t->logical_block_size - 1)) {
if (q->mq_ops)
blk_mq_free_queue(q);
-
- kfree(q->flush_rq);
+ else
+ blk_free_flush_queue(q->fq);
blk_trace_shutdown(q);
switch (ret) {
case BLK_EH_HANDLED:
/* Can we use req->errors here? */
- if (q->mq_ops)
- __blk_mq_complete_request(req);
- else
- __blk_complete_request(req);
+ __blk_complete_request(req);
break;
case BLK_EH_RESET_TIMER:
blk_add_timer(req);
}
}
-void blk_rq_check_expired(struct request *rq, unsigned long *next_timeout,
+static void blk_rq_check_expired(struct request *rq, unsigned long *next_timeout,
unsigned int *next_set)
{
if (time_after_eq(jiffies, rq->deadline)) {
if (blk_mark_rq_complete(req))
return;
blk_delete_timer(req);
- blk_rq_timed_out(req);
+ if (req->q->mq_ops)
+ blk_mq_rq_timed_out(req, false);
+ else
+ blk_rq_timed_out(req);
}
EXPORT_SYMBOL_GPL(blk_abort_request);
struct request_queue *q = req->q;
unsigned long expiry;
- if (!q->rq_timed_out_fn)
+ /* blk-mq has its own handler, so we don't need ->rq_timed_out_fn */
+ if (!q->mq_ops && !q->rq_timed_out_fn)
return;
BUG_ON(!list_empty(&req->timeout_list));
#define BLK_INTERNAL_H
#include <linux/idr.h>
+#include <linux/blk-mq.h>
+#include "blk-mq.h"
/* Amount of time in which a process may batch requests */
#define BLK_BATCH_TIME (HZ/50UL)
/* Max future timer expiry for timeouts */
#define BLK_MAX_TIMEOUT (5 * HZ)
+struct blk_flush_queue {
+ unsigned int flush_queue_delayed:1;
+ unsigned int flush_pending_idx:1;
+ unsigned int flush_running_idx:1;
+ unsigned long flush_pending_since;
+ struct list_head flush_queue[2];
+ struct list_head flush_data_in_flight;
+ struct request *flush_rq;
+ spinlock_t mq_flush_lock;
+};
+
extern struct kmem_cache *blk_requestq_cachep;
extern struct kmem_cache *request_cachep;
extern struct kobj_type blk_queue_ktype;
extern struct ida blk_queue_ida;
+static inline struct blk_flush_queue *blk_get_flush_queue(
+ struct request_queue *q, struct blk_mq_ctx *ctx)
+{
+ struct blk_mq_hw_ctx *hctx;
+
+ if (!q->mq_ops)
+ return q->fq;
+
+ hctx = q->mq_ops->map_queue(q, ctx->cpu);
+
+ return hctx->fq;
+}
+
static inline void __blk_get_queue(struct request_queue *q)
{
kobject_get(&q->kobj);
}
+struct blk_flush_queue *blk_alloc_flush_queue(struct request_queue *q,
+ int node, int cmd_size);
+void blk_free_flush_queue(struct blk_flush_queue *q);
+
int blk_init_rl(struct request_list *rl, struct request_queue *q,
gfp_t gfp_mask);
void blk_exit_rl(struct request_list *rl);
unsigned int nr_bytes, unsigned int bidi_bytes);
void blk_rq_timed_out_timer(unsigned long data);
-void blk_rq_check_expired(struct request *rq, unsigned long *next_timeout,
- unsigned int *next_set);
unsigned long blk_rq_timeout(unsigned long timeout);
void blk_add_timer(struct request *req);
void blk_delete_timer(struct request *);
static inline struct request *__elv_next_request(struct request_queue *q)
{
struct request *rq;
+ struct blk_flush_queue *fq = blk_get_flush_queue(q, NULL);
while (1) {
if (!list_empty(&q->queue_head)) {
* should be restarted later. Please see flush_end_io() for
* details.
*/
- if (q->flush_pending_idx != q->flush_running_idx &&
+ if (fq->flush_pending_idx != fq->flush_running_idx &&
!queue_flush_queueable(q)) {
- q->flush_queue_delayed = 1;
+ fq->flush_queue_delayed = 1;
return NULL;
}
if (unlikely(blk_queue_bypass(q)) ||
* map scatter-gather elements separately and string them to request
*/
rq = blk_get_request(q, rw, GFP_KERNEL);
- if (!rq)
- return ERR_PTR(-ENOMEM);
+ if (IS_ERR(rq))
+ return rq;
blk_rq_set_block_pc(rq);
ret = blk_fill_sgv4_hdr_rq(q, rq, hdr, bd, has_write_perm);
}
next_rq = blk_get_request(q, READ, GFP_KERNEL);
- if (!next_rq) {
- ret = -ENOMEM;
+ if (IS_ERR(next_rq)) {
+ ret = PTR_ERR(next_rq);
+ next_rq = NULL;
goto out;
}
rq->next_rq = next_rq;
struct cfq_ttime ttime;
int ioprio; /* the current ioprio */
#ifdef CONFIG_CFQ_GROUP_IOSCHED
- uint64_t blkcg_id; /* the current blkcg ID */
+ uint64_t blkcg_serial_nr; /* the current blkcg serial */
#endif
};
{
struct cfq_data *cfqd = cic_to_cfqd(cic);
struct cfq_queue *sync_cfqq;
- uint64_t id;
+ uint64_t serial_nr;
rcu_read_lock();
- id = bio_blkcg(bio)->id;
+ serial_nr = bio_blkcg(bio)->css.serial_nr;
rcu_read_unlock();
/*
* Check whether blkcg has changed. The condition may trigger
* spuriously on a newly created cic but there's no harm.
*/
- if (unlikely(!cfqd) || likely(cic->blkcg_id == id))
+ if (unlikely(!cfqd) || likely(cic->blkcg_serial_nr == serial_nr))
return;
sync_cfqq = cic_to_cfqq(cic, 1);
cfq_put_queue(sync_cfqq);
}
- cic->blkcg_id = id;
+ cic->blkcg_serial_nr = serial_nr;
}
#else
static inline void check_blkcg_changed(struct cfq_io_cq *cic, struct bio *bio) { }
if (!arg)
return -EINVAL;
bdi = blk_get_backing_dev_info(bdev);
- if (bdi == NULL)
- return -ENOTTY;
return compat_put_long(arg,
(bdi->ra_pages * PAGE_CACHE_SIZE) / 512);
case BLKROGET: /* compatible */
if (!capable(CAP_SYS_ADMIN))
return -EACCES;
bdi = blk_get_backing_dev_info(bdev);
- if (bdi == NULL)
- return -ENOTTY;
bdi->ra_pages = (arg * 512) / PAGE_CACHE_SIZE;
return 0;
case BLKGETSIZE:
if (!arg)
return -EINVAL;
bdi = blk_get_backing_dev_info(bdev);
- if (bdi == NULL)
- return -ENOTTY;
return put_long(arg, (bdi->ra_pages * PAGE_CACHE_SIZE) / 512);
case BLKROGET:
return put_int(arg, bdev_read_only(bdev) != 0);
if(!capable(CAP_SYS_ADMIN))
return -EACCES;
bdi = blk_get_backing_dev_info(bdev);
- if (bdi == NULL)
- return -ENOTTY;
bdi->ra_pages = (arg * 512) / PAGE_CACHE_SIZE;
return 0;
case BLKBSZSET:
be32_to_cpu(part->start_block) * (secsize/512),
be32_to_cpu(part->block_count) * (secsize/512));
- if (!strnicmp(part->type, "Linux_RAID", 10))
+ if (!strncasecmp(part->type, "Linux_RAID", 10))
state->parts[slot].flags = ADDPART_FLAG_RAID;
#ifdef CONFIG_PPC_PMAC
/*
goodness++;
if (strcasecmp(part->type, "Apple_UNIX_SVR2") == 0
- || (strnicmp(part->type, "Linux", 5) == 0
+ || (strncasecmp(part->type, "Linux", 5) == 0
&& strcasecmp(part->type, "Linux_swap") != 0)) {
int i, l;
if (strcmp(part->name, "/") == 0)
goodness++;
for (i = 0; i <= l - 4; ++i) {
- if (strnicmp(part->name + i, "root",
+ if (strncasecmp(part->name + i, "root",
4) == 0) {
goodness += 2;
break;
}
}
- if (strnicmp(part->name, "swap", 4) == 0)
+ if (strncasecmp(part->name, "swap", 4) == 0)
goodness--;
}
ret = -ENOMEM;
rq = blk_get_request(q, writing ? WRITE : READ, GFP_KERNEL);
- if (!rq)
- goto out;
+ if (IS_ERR(rq))
+ return PTR_ERR(rq);
blk_rq_set_block_pc(rq);
if (hdr->cmd_len > BLK_MAX_CDB) {
kfree(rq->cmd);
out_put_request:
blk_put_request(rq);
-out:
return ret;
}
}
rq = blk_get_request(q, in_len ? WRITE : READ, __GFP_WAIT);
- if (!rq) {
- err = -ENOMEM;
+ if (IS_ERR(rq)) {
+ err = PTR_ERR(rq);
goto error;
}
blk_rq_set_block_pc(rq);
int err;
rq = blk_get_request(q, WRITE, __GFP_WAIT);
+ if (IS_ERR(rq))
+ return PTR_ERR(rq);
blk_rq_set_block_pc(rq);
rq->timeout = BLK_DEFAULT_SG_TIMEOUT;
rq->cmd[0] = cmd;
--- /dev/null
+/*
+ * t10_pi.c - Functions for generating and verifying T10 Protection
+ * Information.
+ *
+ * Copyright (C) 2007, 2008, 2014 Oracle Corporation
+ * Written by: Martin K. Petersen <martin.petersen@oracle.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; see the file COPYING. If not, write to
+ * the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139,
+ * USA.
+ *
+ */
+
+#include <linux/t10-pi.h>
+#include <linux/blkdev.h>
+#include <linux/crc-t10dif.h>
+#include <net/checksum.h>
+
+typedef __be16 (csum_fn) (void *, unsigned int);
+
+static const __be16 APP_ESCAPE = (__force __be16) 0xffff;
+static const __be32 REF_ESCAPE = (__force __be32) 0xffffffff;
+
+static __be16 t10_pi_crc_fn(void *data, unsigned int len)
+{
+ return cpu_to_be16(crc_t10dif(data, len));
+}
+
+static __be16 t10_pi_ip_fn(void *data, unsigned int len)
+{
+ return (__force __be16)ip_compute_csum(data, len);
+}
+
+/*
+ * Type 1 and Type 2 protection use the same format: 16 bit guard tag,
+ * 16 bit app tag, 32 bit reference tag. Type 3 does not define the ref
+ * tag.
+ */
+static int t10_pi_generate(struct blk_integrity_iter *iter, csum_fn *fn,
+ unsigned int type)
+{
+ unsigned int i;
+
+ for (i = 0 ; i < iter->data_size ; i += iter->interval) {
+ struct t10_pi_tuple *pi = iter->prot_buf;
+
+ pi->guard_tag = fn(iter->data_buf, iter->interval);
+ pi->app_tag = 0;
+
+ if (type == 1)
+ pi->ref_tag = cpu_to_be32(lower_32_bits(iter->seed));
+ else
+ pi->ref_tag = 0;
+
+ iter->data_buf += iter->interval;
+ iter->prot_buf += sizeof(struct t10_pi_tuple);
+ iter->seed++;
+ }
+
+ return 0;
+}
+
+static int t10_pi_verify(struct blk_integrity_iter *iter, csum_fn *fn,
+ unsigned int type)
+{
+ unsigned int i;
+
+ for (i = 0 ; i < iter->data_size ; i += iter->interval) {
+ struct t10_pi_tuple *pi = iter->prot_buf;
+ __be16 csum;
+
+ switch (type) {
+ case 1:
+ case 2:
+ if (pi->app_tag == APP_ESCAPE)
+ goto next;
+
+ if (be32_to_cpu(pi->ref_tag) !=
+ lower_32_bits(iter->seed)) {
+ pr_err("%s: ref tag error at location %llu " \
+ "(rcvd %u)\n", iter->disk_name,
+ (unsigned long long)
+ iter->seed, be32_to_cpu(pi->ref_tag));
+ return -EILSEQ;
+ }
+ break;
+ case 3:
+ if (pi->app_tag == APP_ESCAPE &&
+ pi->ref_tag == REF_ESCAPE)
+ goto next;
+ break;
+ }
+
+ csum = fn(iter->data_buf, iter->interval);
+
+ if (pi->guard_tag != csum) {
+ pr_err("%s: guard tag error at sector %llu " \
+ "(rcvd %04x, want %04x)\n", iter->disk_name,
+ (unsigned long long)iter->seed,
+ be16_to_cpu(pi->guard_tag), be16_to_cpu(csum));
+ return -EILSEQ;
+ }
+
+next:
+ iter->data_buf += iter->interval;
+ iter->prot_buf += sizeof(struct t10_pi_tuple);
+ iter->seed++;
+ }
+
+ return 0;
+}
+
+static int t10_pi_type1_generate_crc(struct blk_integrity_iter *iter)
+{
+ return t10_pi_generate(iter, t10_pi_crc_fn, 1);
+}
+
+static int t10_pi_type1_generate_ip(struct blk_integrity_iter *iter)
+{
+ return t10_pi_generate(iter, t10_pi_ip_fn, 1);
+}
+
+static int t10_pi_type1_verify_crc(struct blk_integrity_iter *iter)
+{
+ return t10_pi_verify(iter, t10_pi_crc_fn, 1);
+}
+
+static int t10_pi_type1_verify_ip(struct blk_integrity_iter *iter)
+{
+ return t10_pi_verify(iter, t10_pi_ip_fn, 1);
+}
+
+static int t10_pi_type3_generate_crc(struct blk_integrity_iter *iter)
+{
+ return t10_pi_generate(iter, t10_pi_crc_fn, 3);
+}
+
+static int t10_pi_type3_generate_ip(struct blk_integrity_iter *iter)
+{
+ return t10_pi_generate(iter, t10_pi_ip_fn, 3);
+}
+
+static int t10_pi_type3_verify_crc(struct blk_integrity_iter *iter)
+{
+ return t10_pi_verify(iter, t10_pi_crc_fn, 3);
+}
+
+static int t10_pi_type3_verify_ip(struct blk_integrity_iter *iter)
+{
+ return t10_pi_verify(iter, t10_pi_ip_fn, 3);
+}
+
+struct blk_integrity t10_pi_type1_crc = {
+ .name = "T10-DIF-TYPE1-CRC",
+ .generate_fn = t10_pi_type1_generate_crc,
+ .verify_fn = t10_pi_type1_verify_crc,
+ .tuple_size = sizeof(struct t10_pi_tuple),
+ .tag_size = 0,
+};
+EXPORT_SYMBOL(t10_pi_type1_crc);
+
+struct blk_integrity t10_pi_type1_ip = {
+ .name = "T10-DIF-TYPE1-IP",
+ .generate_fn = t10_pi_type1_generate_ip,
+ .verify_fn = t10_pi_type1_verify_ip,
+ .tuple_size = sizeof(struct t10_pi_tuple),
+ .tag_size = 0,
+};
+EXPORT_SYMBOL(t10_pi_type1_ip);
+
+struct blk_integrity t10_pi_type3_crc = {
+ .name = "T10-DIF-TYPE3-CRC",
+ .generate_fn = t10_pi_type3_generate_crc,
+ .verify_fn = t10_pi_type3_verify_crc,
+ .tuple_size = sizeof(struct t10_pi_tuple),
+ .tag_size = 0,
+};
+EXPORT_SYMBOL(t10_pi_type3_crc);
+
+struct blk_integrity t10_pi_type3_ip = {
+ .name = "T10-DIF-TYPE3-IP",
+ .generate_fn = t10_pi_type3_generate_ip,
+ .verify_fn = t10_pi_type3_verify_ip,
+ .tuple_size = sizeof(struct t10_pi_tuple),
+ .tag_size = 0,
+};
+EXPORT_SYMBOL(t10_pi_type3_ip);
.probe = lanai_init_one,
};
-static int __init lanai_module_init(void)
-{
- int x;
-
- x = pci_register_driver(&lanai_driver);
- if (x != 0)
- printk(KERN_ERR DEV_LABEL ": no adapter found\n");
- return x;
-}
-
-static void __exit lanai_module_exit(void)
-{
- /* We'll only get called when all the interfaces are already
- * gone, so there isn't much to do
- */
- DPRINTK("cleanup_module()\n");
- pci_unregister_driver(&lanai_driver);
-}
-
-module_init(lanai_module_init);
-module_exit(lanai_module_exit);
+module_pci_driver(lanai_driver);
MODULE_AUTHOR("Mitchell Blank Jr <mitch@sfgoth.com>");
MODULE_DESCRIPTION("Efficient Networks Speedstream 3010 driver");
if (bio_add_page(bio, device->md_io.page, size, 0) != size)
goto out;
bio->bi_private = device;
- bio->bi_end_io = drbd_md_io_complete;
+ bio->bi_end_io = drbd_md_endio;
bio->bi_rw = rw;
if (!(rw & WRITE) && device->state.disk == D_DISKLESS && device->ldev == NULL)
/* special case, drbd_md_read() during drbd_adm_attach(): no get_ldev */
;
else if (!get_ldev_if_state(device, D_ATTACHING)) {
- /* Corresponding put_ldev in drbd_md_io_complete() */
+ /* Corresponding put_ldev in drbd_md_endio() */
drbd_err(device, "ASSERT FAILED: get_ldev_if_state() == 1 in _drbd_md_sync_page_io()\n");
err = -ENODEV;
goto out;
}
/* bv_page may be a copy, or may be the original */
-static void bm_async_io_complete(struct bio *bio, int error)
+static void drbd_bm_endio(struct bio *bio, int error)
{
struct drbd_bm_aio_ctx *ctx = bio->bi_private;
struct drbd_device *device = ctx->device;
* according to api. Do we want to assert that? */
bio_add_page(bio, page, len, 0);
bio->bi_private = ctx;
- bio->bi_end_io = bm_async_io_complete;
+ bio->bi_end_io = drbd_bm_endio;
if (drbd_insert_fault(device, (rw & WRITE) ? DRBD_FAULT_MD_WR : DRBD_FAULT_MD_RD)) {
bio->bi_rw |= rw;
}
/*
- * We initialize ctx->in_flight to one to make sure bm_async_io_complete
+ * We initialize ctx->in_flight to one to make sure drbd_bm_endio
* will not set ctx->done early, and decrement / test it here. If there
* are still some bios in flight, we need to wait for them here.
* If all IO is done already (or nothing had been submitted), there is
{
struct bm_extent *bme = lc_entry(e, struct bm_extent, lce);
- seq_printf(m, "%5d %s %s %s\n", bme->rs_left,
+ seq_printf(m, "%5d %s %s %s", bme->rs_left,
test_bit(BME_NO_WRITES, &bme->flags) ? "NO_WRITES" : "---------",
test_bit(BME_LOCKED, &bme->flags) ? "LOCKED" : "------",
test_bit(BME_PRIORITY, &bme->flags) ? "PRIORITY" : "--------"
# define __must_hold(x)
#endif
-#define __no_warn(lock, stmt) do { __acquire(lock); stmt; __release(lock); } while (0)
-
/* module parameter, defined in drbd_main.c */
extern unsigned int minor_count;
extern bool disable_sendpage;
/* drbd_worker.c */
/* bi_end_io handlers */
-extern void drbd_md_io_complete(struct bio *bio, int error);
+extern void drbd_md_endio(struct bio *bio, int error);
extern void drbd_peer_request_endio(struct bio *bio, int error);
extern void drbd_request_endio(struct bio *bio, int error);
extern int drbd_worker(struct drbd_thread *thi);
/**
* get_ldev() - Increase the ref count on device->ldev. Returns 0 if there is no ldev
- * @M: DRBD device.
+ * @_device: DRBD device.
+ * @_min_state: Minimum device state required for success.
*
* You have to call put_ldev() when finished working with device->ldev.
*/
-#define get_ldev(M) __cond_lock(local, _get_ldev_if_state(M,D_INCONSISTENT))
-#define get_ldev_if_state(M,MINS) __cond_lock(local, _get_ldev_if_state(M,MINS))
+#define get_ldev_if_state(_device, _min_state) \
+ (_get_ldev_if_state((_device), (_min_state)) ? \
+ ({ __acquire(x); true; }) : false)
+#define get_ldev(_device) get_ldev_if_state(_device, D_INCONSISTENT)
static inline void put_ldev(struct drbd_device *device)
{
- enum drbd_disk_state ds = device->state.disk;
+ enum drbd_disk_state disk_state = device->state.disk;
/* We must check the state *before* the atomic_dec becomes visible,
* or we have a theoretical race where someone hitting zero,
* while state still D_FAILED, will then see D_DISKLESS in the
__release(local);
D_ASSERT(device, i >= 0);
if (i == 0) {
- if (ds == D_DISKLESS)
+ if (disk_state == D_DISKLESS)
/* even internal references gone, safe to destroy */
drbd_device_post_work(device, DESTROY_DISK);
- if (ds == D_FAILED)
+ if (disk_state == D_FAILED)
/* all application IO references gone. */
if (!test_and_set_bit(GOING_DISKLESS, &device->flags))
drbd_device_post_work(device, GO_DISKLESS);
return max;
}
-static void augment_propagate(struct rb_node *rb, struct rb_node *stop)
-{
- while (rb != stop) {
- struct drbd_interval *node = rb_entry(rb, struct drbd_interval, rb);
- sector_t subtree_last = compute_subtree_last(node);
- if (node->end == subtree_last)
- break;
- node->end = subtree_last;
- rb = rb_parent(&node->rb);
- }
-}
-
-static void augment_copy(struct rb_node *rb_old, struct rb_node *rb_new)
-{
- struct drbd_interval *old = rb_entry(rb_old, struct drbd_interval, rb);
- struct drbd_interval *new = rb_entry(rb_new, struct drbd_interval, rb);
-
- new->end = old->end;
-}
-
-static void augment_rotate(struct rb_node *rb_old, struct rb_node *rb_new)
-{
- struct drbd_interval *old = rb_entry(rb_old, struct drbd_interval, rb);
- struct drbd_interval *new = rb_entry(rb_new, struct drbd_interval, rb);
-
- new->end = old->end;
- old->end = compute_subtree_last(old);
-}
-
-static const struct rb_augment_callbacks augment_callbacks = {
- augment_propagate,
- augment_copy,
- augment_rotate,
-};
+RB_DECLARE_CALLBACKS(static, augment_callbacks, struct drbd_interval, rb,
+ sector_t, end, compute_subtree_last);
/**
* drbd_insert_interval - insert a new interval into a tree
drbd_insert_interval(struct rb_root *root, struct drbd_interval *this)
{
struct rb_node **new = &root->rb_node, *parent = NULL;
+ sector_t this_end = this->sector + (this->size >> 9);
BUG_ON(!IS_ALIGNED(this->size, 512));
rb_entry(*new, struct drbd_interval, rb);
parent = *new;
+ if (here->end < this_end)
+ here->end = this_end;
if (this->sector < here->sector)
new = &(*new)->rb_left;
else if (this->sector > here->sector)
return false;
}
+ this->end = this_end;
rb_link_node(&this->rb, parent, new);
rb_insert_augmented(&this->rb, root, &augment_callbacks);
return true;
struct drbd_socket *sock;
struct p_data *p;
unsigned int dp_flags = 0;
- int dgs;
+ int digest_size;
int err;
sock = &peer_device->connection->data;
p = drbd_prepare_command(peer_device, sock);
- dgs = peer_device->connection->integrity_tfm ?
- crypto_hash_digestsize(peer_device->connection->integrity_tfm) : 0;
+ digest_size = peer_device->connection->integrity_tfm ?
+ crypto_hash_digestsize(peer_device->connection->integrity_tfm) : 0;
if (!p)
return -EIO;
/* our digest is still only over the payload.
* TRIM does not carry any payload. */
- if (dgs)
+ if (digest_size)
drbd_csum_bio(peer_device->connection->integrity_tfm, req->master_bio, p + 1);
- err = __send_command(peer_device->connection, device->vnr, sock, P_DATA, sizeof(*p) + dgs, NULL, req->i.size);
+ err = __send_command(peer_device->connection, device->vnr, sock, P_DATA, sizeof(*p) + digest_size, NULL, req->i.size);
if (!err) {
/* For protocol A, we have to memcpy the payload into
* socket buffers, as we may complete right away
* out ok after sending on this side, but does not fit on the
* receiving side, we sure have detected corruption elsewhere.
*/
- if (!(req->rq_state & (RQ_EXP_RECEIVE_ACK | RQ_EXP_WRITE_ACK)) || dgs)
+ if (!(req->rq_state & (RQ_EXP_RECEIVE_ACK | RQ_EXP_WRITE_ACK)) || digest_size)
err = _drbd_send_bio(peer_device, req->master_bio);
else
err = _drbd_send_zc_bio(peer_device, req->master_bio);
/* double check digest, sometimes buffers have been modified in flight. */
- if (dgs > 0 && dgs <= 64) {
+ if (digest_size > 0 && digest_size <= 64) {
/* 64 byte, 512 bit, is the largest digest size
* currently supported in kernel crypto. */
unsigned char digest[64];
drbd_csum_bio(peer_device->connection->integrity_tfm, req->master_bio, digest);
- if (memcmp(p + 1, digest, dgs)) {
+ if (memcmp(p + 1, digest, digest_size)) {
drbd_warn(device,
"Digest mismatch, buffer modified by upper layers during write: %llus +%u\n",
(unsigned long long)req->i.sector, req->i.size);
}
- } /* else if (dgs > 64) {
+ } /* else if (digest_size > 64) {
... Be noisy about digest too large ...
} */
}
struct drbd_socket *sock;
struct p_data *p;
int err;
- int dgs;
+ int digest_size;
sock = &peer_device->connection->data;
p = drbd_prepare_command(peer_device, sock);
- dgs = peer_device->connection->integrity_tfm ?
- crypto_hash_digestsize(peer_device->connection->integrity_tfm) : 0;
+ digest_size = peer_device->connection->integrity_tfm ?
+ crypto_hash_digestsize(peer_device->connection->integrity_tfm) : 0;
if (!p)
return -EIO;
p->block_id = peer_req->block_id;
p->seq_num = 0; /* unused */
p->dp_flags = 0;
- if (dgs)
+ if (digest_size)
drbd_csum_ee(peer_device->connection->integrity_tfm, peer_req, p + 1);
- err = __send_command(peer_device->connection, device->vnr, sock, cmd, sizeof(*p) + dgs, NULL, peer_req->i.size);
+ err = __send_command(peer_device->connection, device->vnr, sock, cmd, sizeof(*p) + digest_size, NULL, peer_req->i.size);
if (!err)
err = _drbd_send_zc_ee(peer_device, peer_req);
mutex_unlock(&sock->mutex); /* locked by drbd_prepare_command() */
(unsigned long) Bit2KB(rs_left >> 10),
(unsigned long) Bit2KB(rs_total >> 10));
else
- seq_printf(seq, "(%lu/%lu)K\n\t",
+ seq_printf(seq, "(%lu/%lu)K",
(unsigned long) Bit2KB(rs_left),
(unsigned long) Bit2KB(rs_total));
+ seq_printf(seq, "\n\t");
+
/* see drivers/md/md.c
* We do not want to overflow, so the order of operands and
* the * 100 / 100 trick are important. We do a +1 to be
struct bio *bio;
struct page *page = peer_req->pages;
sector_t sector = peer_req->i.sector;
- unsigned ds = peer_req->i.size;
+ unsigned data_size = peer_req->i.size;
unsigned n_bios = 0;
- unsigned nr_pages = (ds + PAGE_SIZE -1) >> PAGE_SHIFT;
+ unsigned nr_pages = (data_size + PAGE_SIZE -1) >> PAGE_SHIFT;
int err = -ENOMEM;
if (peer_req->flags & EE_IS_TRIM_USE_ZEROOUT) {
list_add_tail(&peer_req->w.list, &device->active_ee);
spin_unlock_irq(&device->resource->req_lock);
if (blkdev_issue_zeroout(device->ldev->backing_bdev,
- sector, ds >> 9, GFP_NOIO))
+ sector, data_size >> 9, GFP_NOIO))
peer_req->flags |= EE_WAS_ERROR;
drbd_endio_write_sec_final(peer_req);
return 0;
++n_bios;
if (rw & REQ_DISCARD) {
- bio->bi_iter.bi_size = ds;
+ bio->bi_iter.bi_size = data_size;
goto submit;
}
page_chain_for_each(page) {
- unsigned len = min_t(unsigned, ds, PAGE_SIZE);
+ unsigned len = min_t(unsigned, data_size, PAGE_SIZE);
if (!bio_add_page(bio, page, len, 0)) {
/* A single page must always be possible!
* But in case it fails anyways,
}
goto next_bio;
}
- ds -= len;
+ data_size -= len;
sector += len >> 9;
--nr_pages;
}
- D_ASSERT(device, ds == 0);
+ D_ASSERT(device, data_size == 0);
submit:
D_ASSERT(device, page == NULL);
const sector_t capacity = drbd_get_capacity(device->this_bdev);
struct drbd_peer_request *peer_req;
struct page *page;
- int dgs, ds, err;
- unsigned int data_size = pi->size;
+ int digest_size, err;
+ unsigned int data_size = pi->size, ds;
void *dig_in = peer_device->connection->int_dig_in;
void *dig_vv = peer_device->connection->int_dig_vv;
unsigned long *data;
struct p_trim *trim = (pi->cmd == P_TRIM) ? pi->data : NULL;
- dgs = 0;
+ digest_size = 0;
if (!trim && peer_device->connection->peer_integrity_tfm) {
- dgs = crypto_hash_digestsize(peer_device->connection->peer_integrity_tfm);
+ digest_size = crypto_hash_digestsize(peer_device->connection->peer_integrity_tfm);
/*
* FIXME: Receive the incoming digest into the receive buffer
* here, together with its struct p_data?
*/
- err = drbd_recv_all_warn(peer_device->connection, dig_in, dgs);
+ err = drbd_recv_all_warn(peer_device->connection, dig_in, digest_size);
if (err)
return NULL;
- data_size -= dgs;
+ data_size -= digest_size;
}
if (trim) {
ds -= len;
}
- if (dgs) {
+ if (digest_size) {
drbd_csum_ee(peer_device->connection->peer_integrity_tfm, peer_req, dig_vv);
- if (memcmp(dig_in, dig_vv, dgs)) {
+ if (memcmp(dig_in, dig_vv, digest_size)) {
drbd_err(device, "Digest integrity check FAILED: %llus +%u\n",
(unsigned long long)sector, data_size);
drbd_free_peer_req(device, peer_req);
return NULL;
}
}
- device->recv_cnt += data_size>>9;
+ device->recv_cnt += data_size >> 9;
return peer_req;
}
struct bio_vec bvec;
struct bvec_iter iter;
struct bio *bio;
- int dgs, err, expect;
+ int digest_size, err, expect;
void *dig_in = peer_device->connection->int_dig_in;
void *dig_vv = peer_device->connection->int_dig_vv;
- dgs = 0;
+ digest_size = 0;
if (peer_device->connection->peer_integrity_tfm) {
- dgs = crypto_hash_digestsize(peer_device->connection->peer_integrity_tfm);
- err = drbd_recv_all_warn(peer_device->connection, dig_in, dgs);
+ digest_size = crypto_hash_digestsize(peer_device->connection->peer_integrity_tfm);
+ err = drbd_recv_all_warn(peer_device->connection, dig_in, digest_size);
if (err)
return err;
- data_size -= dgs;
+ data_size -= digest_size;
}
/* optimistically update recv_cnt. if receiving fails below,
data_size -= expect;
}
- if (dgs) {
+ if (digest_size) {
drbd_csum_bio(peer_device->connection->peer_integrity_tfm, bio, dig_vv);
- if (memcmp(dig_in, dig_vv, dgs)) {
+ if (memcmp(dig_in, dig_vv, digest_size)) {
drbd_err(peer_device, "Digest integrity check FAILED. Broken NICs?\n");
return -EINVAL;
}
* rv < expected: "woken" by signal during receive
* rv == 0 : "connection shut down by peer"
*/
+received_more:
if (likely(rv > 0)) {
received += rv;
buf += rv;
expect = header_size;
cmd = NULL;
}
+ if (test_bit(SEND_PING, &connection->flags))
+ continue;
+ rv = drbd_recv_short(connection->meta.socket, buf, expect-received, MSG_DONTWAIT);
+ if (rv > 0)
+ goto received_more;
}
if (0) {
? oldest_submit_jif + dt : now + et;
nt = time_before(ent, dt) ? ent : dt;
out:
- spin_unlock_irq(&connection->resource->req_lock);
+ spin_unlock_irq(&device->resource->req_lock);
mod_timer(&device->request_timer, nt);
}
enum drbd_disk_state conn_highest_disk(struct drbd_connection *connection)
{
- enum drbd_disk_state ds = D_DISKLESS;
+ enum drbd_disk_state disk_state = D_DISKLESS;
struct drbd_peer_device *peer_device;
int vnr;
rcu_read_lock();
idr_for_each_entry(&connection->peer_devices, peer_device, vnr) {
struct drbd_device *device = peer_device->device;
- ds = max_t(enum drbd_disk_state, ds, device->state.disk);
+ disk_state = max_t(enum drbd_disk_state, disk_state, device->state.disk);
}
rcu_read_unlock();
- return ds;
+ return disk_state;
}
enum drbd_disk_state conn_lowest_disk(struct drbd_connection *connection)
{
- enum drbd_disk_state ds = D_MASK;
+ enum drbd_disk_state disk_state = D_MASK;
struct drbd_peer_device *peer_device;
int vnr;
rcu_read_lock();
idr_for_each_entry(&connection->peer_devices, peer_device, vnr) {
struct drbd_device *device = peer_device->device;
- ds = min_t(enum drbd_disk_state, ds, device->state.disk);
+ disk_state = min_t(enum drbd_disk_state, disk_state, device->state.disk);
}
rcu_read_unlock();
- return ds;
+ return disk_state;
}
enum drbd_disk_state conn_highest_pdsk(struct drbd_connection *connection)
{
- enum drbd_disk_state ds = D_DISKLESS;
+ enum drbd_disk_state disk_state = D_DISKLESS;
struct drbd_peer_device *peer_device;
int vnr;
rcu_read_lock();
idr_for_each_entry(&connection->peer_devices, peer_device, vnr) {
struct drbd_device *device = peer_device->device;
- ds = max_t(enum drbd_disk_state, ds, device->state.pdsk);
+ disk_state = max_t(enum drbd_disk_state, disk_state, device->state.pdsk);
}
rcu_read_unlock();
- return ds;
+ return disk_state;
}
enum drbd_conns conn_lowest_conn(struct drbd_connection *connection)
static int make_resync_request(struct drbd_device *, int);
/* endio handlers:
- * drbd_md_io_complete (defined here)
+ * drbd_md_endio (defined here)
* drbd_request_endio (defined here)
* drbd_peer_request_endio (defined here)
- * bm_async_io_complete (defined in drbd_bitmap.c)
+ * drbd_bm_endio (defined in drbd_bitmap.c)
*
* For all these callbacks, note the following:
* The callbacks will be called in irq context by the IDE drivers,
/* used for synchronous meta data and bitmap IO
* submitted by drbd_md_sync_page_io()
*/
-void drbd_md_io_complete(struct bio *bio, int error)
+void drbd_md_endio(struct bio *bio, int error)
{
struct drbd_device *device;
device->resync = NULL;
lc_destroy(device->act_log);
device->act_log = NULL;
- __no_warn(local,
- drbd_free_ldev(device->ldev);
- device->ldev = NULL;);
+
+ __acquire(local);
+ drbd_free_ldev(device->ldev);
+ device->ldev = NULL;
+ __release(local);
+
clear_bit(GOING_DISKLESS, &device->flags);
wake_up(&device->misc_wait);
}
++(*cb_nr);
}
-#define WORK_PENDING(work_bit, todo) (todo & (1UL << work_bit))
static void do_device_work(struct drbd_device *device, const unsigned long todo)
{
- if (WORK_PENDING(MD_SYNC, todo))
+ if (test_bit(MD_SYNC, &todo))
do_md_sync(device);
- if (WORK_PENDING(RS_DONE, todo) ||
- WORK_PENDING(RS_PROGRESS, todo))
- update_on_disk_bitmap(device, WORK_PENDING(RS_DONE, todo));
- if (WORK_PENDING(GO_DISKLESS, todo))
+ if (test_bit(RS_DONE, &todo) ||
+ test_bit(RS_PROGRESS, &todo))
+ update_on_disk_bitmap(device, test_bit(RS_DONE, &todo));
+ if (test_bit(GO_DISKLESS, &todo))
go_diskless(device);
- if (WORK_PENDING(DESTROY_DISK, todo))
+ if (test_bit(DESTROY_DISK, &todo))
drbd_ldev_destroy(device);
- if (WORK_PENDING(RS_START, todo))
+ if (test_bit(RS_START, &todo))
do_start_resync(device);
}
return !list_empty(work_list);
}
-static bool dequeue_work_item(struct drbd_work_queue *queue, struct list_head *work_list)
-{
- spin_lock_irq(&queue->q_lock);
- if (!list_empty(&queue->q))
- list_move(queue->q.next, work_list);
- spin_unlock_irq(&queue->q_lock);
- return !list_empty(work_list);
-}
-
static void wait_for_work(struct drbd_connection *connection, struct list_head *work_list)
{
DEFINE_WAIT(wait);
struct net_conf *nc;
int uncork, cork;
- dequeue_work_item(&connection->sender_work, work_list);
+ dequeue_work_batch(&connection->sender_work, work_list);
if (!list_empty(work_list))
return;
prepare_to_wait(&connection->sender_work.q_wait, &wait, TASK_INTERRUPTIBLE);
spin_lock_irq(&connection->resource->req_lock);
spin_lock(&connection->sender_work.q_lock); /* FIXME get rid of this one? */
- /* dequeue single item only,
- * we still use drbd_queue_work_front() in some places */
if (!list_empty(&connection->sender_work.q))
list_splice_tail_init(&connection->sender_work.q, work_list);
spin_unlock(&connection->sender_work.q_lock); /* FIXME get rid of this one? */
if (get_t_state(thi) != RUNNING)
break;
- while (!list_empty(&work_list)) {
+ if (!list_empty(&work_list)) {
w = list_first_entry(&work_list, struct drbd_work, list);
list_del_init(&w->list);
update_worker_timing_details(connection, w->cb);
update_worker_timing_details(connection, do_unqueued_work);
do_unqueued_work(connection);
}
- while (!list_empty(&work_list)) {
+ if (!list_empty(&work_list)) {
w = list_first_entry(&work_list, struct drbd_work, list);
list_del_init(&w->list);
update_worker_timing_details(connection, w->cb);
w->cb(w, 1);
- }
- dequeue_work_batch(&connection->sender_work, &work_list);
+ } else
+ dequeue_work_batch(&connection->sender_work, &work_list);
} while (!list_empty(&work_list) || test_bit(DEVICE_WORK_PENDING, &connection->flags));
rcu_read_lock();
.getgeo = hd_getgeo,
};
-/*
- * This is the hard disk IRQ description. The IRQF_DISABLED in sa_flags
- * means we run the IRQ-handler with interrupts disabled: this is bad for
- * interrupt latency, but anything else has led to problems on some
- * machines.
- *
- * We enable interrupts in some of the routines after making sure it's
- * safe.
- */
-
static int __init hd_init(void)
{
int drive;
p->cyl, p->head, p->sect);
}
- if (request_irq(HD_IRQ, hd_interrupt, IRQF_DISABLED, "hd", NULL)) {
+ if (request_irq(HD_IRQ, hd_interrupt, 0, "hd", NULL)) {
printk("hd: unable to get IRQ%d for the hard disk driver\n",
HD_IRQ);
goto out1;
if (unlikely(cmd->unaligned))
up(&port->cmd_slot_unal);
- blk_mq_end_io(rq, status ? -EIO : 0);
+ blk_mq_end_request(rq, status ? -EIO : 0);
}
/*
int err;
err = mtip_send_trim(dd, blk_rq_pos(rq), blk_rq_sectors(rq));
- blk_mq_end_io(rq, err);
+ blk_mq_end_request(rq, err);
return 0;
}
return false;
}
-static int mtip_queue_rq(struct blk_mq_hw_ctx *hctx, struct request *rq)
+static int mtip_queue_rq(struct blk_mq_hw_ctx *hctx, struct request *rq,
+ bool last)
{
int ret;
if (unlikely(mtip_check_unal_depth(hctx, rq)))
return BLK_MQ_RQ_QUEUE_BUSY;
+ blk_mq_start_request(rq);
+
ret = mtip_submit_request(hctx, rq);
if (likely(!ret))
return BLK_MQ_RQ_QUEUE_OK;
/* Set device limits. */
set_bit(QUEUE_FLAG_NONROT, &dd->queue->queue_flags);
+ clear_bit(QUEUE_FLAG_ADD_RANDOM, &dd->queue->queue_flags);
blk_queue_max_segments(dd->queue, MTIP_MAX_SG);
blk_queue_physical_block_size(dd->queue, 4096);
blk_queue_max_hw_sectors(dd->queue, 0xffff);
* Tell the block layer that we are not a rotational device
*/
queue_flag_set_unlocked(QUEUE_FLAG_NONROT, disk->queue);
+ queue_flag_clear_unlocked(QUEUE_FLAG_ADD_RANDOM, disk->queue);
disk->queue->limits.discard_granularity = 512;
disk->queue->limits.max_discard_sectors = UINT_MAX;
disk->queue->limits.discard_zeroes_data = 0;
{
switch (queue_mode) {
case NULL_Q_MQ:
- blk_mq_end_io(cmd->rq, 0);
+ blk_mq_end_request(cmd->rq, 0);
return;
case NULL_Q_RQ:
INIT_LIST_HEAD(&cmd->rq->queuelist);
}
}
-static int null_queue_rq(struct blk_mq_hw_ctx *hctx, struct request *rq)
+static int null_queue_rq(struct blk_mq_hw_ctx *hctx, struct request *rq,
+ bool last)
{
struct nullb_cmd *cmd = blk_mq_rq_to_pdu(rq);
cmd->rq = rq;
cmd->nq = hctx->driver_data;
+ blk_mq_start_request(rq);
+
null_handle_cmd(cmd);
return BLK_MQ_RQ_QUEUE_OK;
}
nullb->q->queuedata = nullb;
queue_flag_set_unlocked(QUEUE_FLAG_NONROT, nullb->q);
+ queue_flag_clear_unlocked(QUEUE_FLAG_ADD_RANDOM, nullb->q);
disk = nullb->disk = alloc_disk_node(1, home_node);
if (!disk) {
ns->queue->queue_flags = QUEUE_FLAG_DEFAULT;
queue_flag_set_unlocked(QUEUE_FLAG_NOMERGES, ns->queue);
queue_flag_set_unlocked(QUEUE_FLAG_NONROT, ns->queue);
+ queue_flag_clear_unlocked(QUEUE_FLAG_ADD_RANDOM, ns->queue);
blk_queue_make_request(ns->queue, nvme_make_request);
ns->dev = dev;
ns->queue->queuedata = ns;
int err = 0;
rq = blk_get_request(disk->gd->queue, READ, __GFP_WAIT);
+ if (IS_ERR(rq))
+ return PTR_ERR(rq);
rq->cmd_type = REQ_TYPE_SPECIAL;
rq->special = func;
rq = blk_get_request(q, (cgc->data_direction == CGC_DATA_WRITE) ?
WRITE : READ, __GFP_WAIT);
+ if (IS_ERR(rq))
+ return PTR_ERR(rq);
blk_rq_set_block_pc(rq);
if (cgc->buflen) {
"Failed to enable MSI\n");
}
- st = request_irq(dev->irq, rsxx_isr, IRQF_DISABLED | IRQF_SHARED,
+ st = request_irq(dev->irq, rsxx_isr, IRQF_SHARED,
DRIVER_NAME, card);
if (st) {
dev_err(CARD_TO_DEV(card),
blk_queue_physical_block_size(card->queue, RSXX_HW_BLK_SIZE);
queue_flag_set_unlocked(QUEUE_FLAG_NONROT, card->queue);
+ queue_flag_clear_unlocked(QUEUE_FLAG_ADD_RANDOM, card->queue);
if (rsxx_discard_supported(card)) {
queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, card->queue);
blk_queue_max_discard_sectors(card->queue,
q->limits.discard_zeroes_data = 1;
queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, q);
queue_flag_set_unlocked(QUEUE_FLAG_NONROT, q);
+ queue_flag_clear_unlocked(QUEUE_FLAG_ADD_RANDOM, q);
spin_lock_irqsave(&skdev->lock, flags);
pr_debug("%s:%s:%d stopping %s queue\n",
return NULL;
rq = blk_get_request(host->oob_q, WRITE /* bogus */, GFP_KERNEL);
- if (!rq) {
+ if (IS_ERR(rq)) {
spin_lock_irqsave(&host->lock, flags);
carm_put_request(host, crq);
spin_unlock_irqrestore(&host->lock, flags);
/* Process context for config space updates */
struct work_struct config_work;
- /* Lock for config space updates */
- struct mutex config_lock;
-
- /* enable config space updates */
- bool config_enable;
-
/* What host tells us, plus 2 for header & tailer. */
unsigned int sg_elems;
req->errors = (error != 0);
}
- blk_mq_end_io(req, error);
+ blk_mq_end_request(req, error);
}
static void virtblk_done(struct virtqueue *vq)
spin_unlock_irqrestore(&vblk->vqs[qid].lock, flags);
}
-static int virtio_queue_rq(struct blk_mq_hw_ctx *hctx, struct request *req)
+static int virtio_queue_rq(struct blk_mq_hw_ctx *hctx, struct request *req,
+ bool last)
{
struct virtio_blk *vblk = hctx->queue->queuedata;
struct virtblk_req *vbr = blk_mq_rq_to_pdu(req);
unsigned long flags;
unsigned int num;
int qid = hctx->queue_num;
- const bool last = (req->cmd_flags & REQ_END) != 0;
int err;
bool notify = false;
}
}
+ blk_mq_start_request(req);
+
num = blk_rq_map_sg(hctx->queue, vbr->req, vbr->sg);
if (num) {
if (rq_data_dir(vbr->req) == WRITE)
char *envp[] = { "RESIZE=1", NULL };
u64 capacity, size;
- mutex_lock(&vblk->config_lock);
- if (!vblk->config_enable)
- goto done;
-
/* Host must always specify the capacity. */
virtio_cread(vdev, struct virtio_blk_config, capacity, &capacity);
set_capacity(vblk->disk, capacity);
revalidate_disk(vblk->disk);
kobject_uevent_env(&disk_to_dev(vblk->disk)->kobj, KOBJ_CHANGE, envp);
-done:
- mutex_unlock(&vblk->config_lock);
}
static void virtblk_config_changed(struct virtio_device *vdev)
vblk->vdev = vdev;
vblk->sg_elems = sg_elems;
- mutex_init(&vblk->config_lock);
INIT_WORK(&vblk->config_work, virtblk_config_changed_work);
- vblk->config_enable = true;
err = init_vq(vblk);
if (err)
if (!err && opt_io_size)
blk_queue_io_opt(q, blk_size * opt_io_size);
+ virtio_device_ready(vdev);
+
add_disk(vblk->disk);
err = device_create_file(disk_to_dev(vblk->disk), &dev_attr_serial);
if (err)
int index = vblk->index;
int refc;
- /* Prevent config work handler from accessing the device. */
- mutex_lock(&vblk->config_lock);
- vblk->config_enable = false;
- mutex_unlock(&vblk->config_lock);
+ /* Make sure no work handler is accessing the device. */
+ flush_work(&vblk->config_work);
del_gendisk(vblk->disk);
blk_cleanup_queue(vblk->disk->queue);
/* Stop all the virtqueues. */
vdev->config->reset(vdev);
- flush_work(&vblk->config_work);
-
refc = atomic_read(&disk_to_dev(vblk->disk)->kobj.kref.refcount);
put_disk(vblk->disk);
vdev->config->del_vqs(vdev);
/* Ensure we don't receive any more interrupts */
vdev->config->reset(vdev);
- /* Prevent config work handler from accessing the device. */
- mutex_lock(&vblk->config_lock);
- vblk->config_enable = false;
- mutex_unlock(&vblk->config_lock);
-
+ /* Make sure no work handler is accessing the device. */
flush_work(&vblk->config_work);
blk_mq_stop_hw_queues(vblk->disk->queue);
struct virtio_blk *vblk = vdev->priv;
int ret;
- vblk->config_enable = true;
ret = init_vq(vdev->priv);
- if (!ret)
- blk_mq_start_stopped_hw_queues(vblk->disk->queue, true);
+ if (ret)
+ return ret;
+
+ virtio_device_ready(vdev);
- return ret;
+ blk_mq_start_stopped_hw_queues(vblk->disk->queue, true);
+ return 0;
}
#endif
BUG_ON(new_map_idx >= segs_to_map);
if (unlikely(map[new_map_idx].status != 0)) {
pr_debug(DRV_PFX "invalid buffer -- could not remap it\n");
+ put_free_pages(blkif, &pages[seg_idx]->page, 1);
pages[seg_idx]->handle = BLKBACK_INVALID_HANDLE;
ret |= 1;
goto next;
blkif->blk_rings.common.sring = NULL;
}
+ /* Remove all persistent grants and the cache of ballooned pages. */
+ xen_blkbk_free_caches(blkif);
+
return 0;
}
xen_blkif_disconnect(blkif);
xen_vbd_free(&blkif->vbd);
- /* Remove all persistent grants and the cache of ballooned pages. */
- xen_blkbk_free_caches(blkif);
-
/* Make sure everything is drained before shutting down */
BUG_ON(blkif->persistent_gnt_c != 0);
BUG_ON(atomic_read(&blkif->persistent_gnt_in_use) != 0);
notify_remote_via_irq(info->irq);
}
+static inline bool blkif_request_flush_valid(struct request *req,
+ struct blkfront_info *info)
+{
+ return ((req->cmd_type != REQ_TYPE_FS) ||
+ ((req->cmd_flags & (REQ_FLUSH | REQ_FUA)) &&
+ !info->flush_op));
+}
+
/*
* do_blkif_request
* read a block; request is in a request queue
blk_start_request(req);
- if ((req->cmd_type != REQ_TYPE_FS) ||
- ((req->cmd_flags & (REQ_FLUSH | REQ_FUA)) &&
- !info->flush_op)) {
+ if (blkif_request_flush_valid(req, info)) {
__blk_end_request_all(req, -EIO);
continue;
}
set_capacity(zram->disk, 0);
/* zram devices sort of resembles non-rotational disks */
queue_flag_set_unlocked(QUEUE_FLAG_NONROT, zram->disk->queue);
+ queue_flag_clear_unlocked(QUEUE_FLAG_ADD_RANDOM, zram->disk->queue);
/*
* To ensure that we always get PAGE_SIZE aligned
* and n*PAGE_SIZED sized I/O requests.
len = nr * CD_FRAMESIZE_RAW;
rq = blk_get_request(q, READ, GFP_KERNEL);
- if (!rq) {
- ret = -ENOMEM;
+ if (IS_ERR(rq)) {
+ ret = PTR_ERR(rq);
break;
}
blk_rq_set_block_pc(rq);
vi->index = index = ida_simple_get(&rng_index_ida, 0, 0, GFP_KERNEL);
if (index < 0) {
- kfree(vi);
- return index;
+ err = index;
+ goto err_ida;
}
sprintf(vi->name, "virtio_rng.%d", index);
init_completion(&vi->have_data);
vi->vq = virtio_find_single_vq(vdev, random_recv_done, "input");
if (IS_ERR(vi->vq)) {
err = PTR_ERR(vi->vq);
- vi->vq = NULL;
- kfree(vi);
- ida_simple_remove(&rng_index_ida, index);
- return err;
+ goto err_find;
}
return 0;
+
+err_find:
+ ida_simple_remove(&rng_index_ida, index);
+err_ida:
+ kfree(vi);
+ return err;
}
static void remove_common(struct virtio_device *vdev)
spin_lock_init(&port->outvq_lock);
init_waitqueue_head(&port->waitqueue);
+ virtio_device_ready(portdev->vdev);
+
/* Fill the in_vq with buffers so the host can send us data. */
nr_added_bufs = fill_queue(port->in_vq, &port->inbuf_lock);
if (!nr_added_bufs) {
if (ret)
return ret;
+ virtio_device_ready(portdev->vdev);
+
if (use_multiport(portdev))
fill_queue(portdev->c_ivq, &portdev->c_ivq_lock);
#define ATIIXP_IDE_UDMA_CONTROL 0x54
#define ATIIXP_IDE_UDMA_MODE 0x56
-typedef struct {
+struct atiixp_ide_timing {
u8 command_width;
u8 recover_width;
-} atiixp_ide_timing;
+};
-static atiixp_ide_timing pio_timing[] = {
+static struct atiixp_ide_timing pio_timing[] = {
{ 0x05, 0x0d },
{ 0x04, 0x07 },
{ 0x03, 0x04 },
{ 0x02, 0x00 },
};
-static atiixp_ide_timing mdma_timing[] = {
+static struct atiixp_ide_timing mdma_timing[] = {
{ 0x07, 0x07 },
{ 0x02, 0x01 },
{ 0x02, 0x00 },
printk(KERN_INFO "%s: max request size: %dKiB\n", drive->name,
queue_max_sectors(q) / 2);
- if (ata_id_is_ssd(id))
+ if (ata_id_is_ssd(id)) {
queue_flag_set_unlocked(QUEUE_FLAG_NONROT, q);
+ queue_flag_clear_unlocked(QUEUE_FLAG_ADD_RANDOM, q);
+ }
/* calculate drive capacity, and select LBA if possible */
ide_disk_get_capacity(drive);
* timeout has expired, so power management will be reenabled.
*/
rq = blk_get_request(q, READ, GFP_NOWAIT);
- if (unlikely(!rq))
+ if (IS_ERR(rq))
goto out;
rq->cmd[0] = REQ_UNPARK_HEADS;
struct sk_buff *skb;
size_t len;
- capi_cmsg2message(cmsg, cmsg->buf);
+ if (capi_cmsg2message(cmsg, cmsg->buf)) {
+ printk(KERN_ERR "capidrv::send_message: parser failure\n");
+ return;
+ }
len = CAPIMSG_LEN(cmsg->buf);
skb = alloc_skb(len, GFP_ATOMIC);
if (!skb) {
static void capidrv_recv_message(struct capi20_appl *ap, struct sk_buff *skb)
{
- capi_message2cmsg(&s_cmsg, skb->data);
+ if (capi_message2cmsg(&s_cmsg, skb->data)) {
+ printk(KERN_ERR "capidrv: applid=%d: received invalid message\n",
+ ap->applid);
+ kfree_skb(skb);
+ return;
+ }
if (debugmode > 3) {
_cdebbuf *cdb = capi_cmsg2str(&s_cmsg);
NULL, /* Useruserdata */
NULL /* Facilitydataarray */
);
- capi_cmsg2message(&cmdcmsg, cmdcmsg.buf);
+ if (capi_cmsg2message(&cmdcmsg, cmdcmsg.buf)) {
+ printk(KERN_ERR "capidrv-%d: capidrv_command: parser failure\n",
+ card->contrnr);
+ return -EINVAL;
+ }
plci_change_state(card, bchan->plcip, EV_PLCI_CONNECT_RESP);
send_message(card, &cmdcmsg);
return 0;
if (capidrv_add_ack(nccip, datahandle, doack ? (int)skb->len : -1) < 0)
return 0;
- capi_cmsg2message(&sendcmsg, sendcmsg.buf);
+ if (capi_cmsg2message(&sendcmsg, sendcmsg.buf)) {
+ printk(KERN_ERR "capidrv-%d: if_sendbuf: parser failure\n",
+ card->contrnr);
+ return -EINVAL;
+ }
msglen = CAPIMSG_LEN(sendcmsg.buf);
if (skb_headroom(skb) < msglen) {
struct sk_buff *nskb = skb_realloc_headroom(skb, msglen);
c = 0x9 + (c & 0x0f);
else if (c == 0x41)
c = 0x9 + 0x1;
+ if (c > 0x18)
+ c = 0x00;
return (sc & 3) * (0x9 + 0x9) + c;
}
+/**
+ * capi_cmd2par() - find parameter string for CAPI 2.0 command/subcommand
+ * @cmd: command number
+ * @subcmd: subcommand number
+ *
+ * Return value: static string, NULL if command/subcommand unknown
+ */
+
+static unsigned char *capi_cmd2par(u8 cmd, u8 subcmd)
+{
+ return cpars[command_2_index(cmd, subcmd)];
+}
+
/*-------------------------------------------------------*/
#define TYP (cdef[cmsg->par[cmsg->p]].typ)
#define OFF (((u8 *)cmsg) + cdef[cmsg->par[cmsg->p]].off)
cmsg->m = msg;
cmsg->l = 8;
cmsg->p = 0;
- cmsg->par = cpars[command_2_index(cmsg->Command, cmsg->Subcommand)];
+ cmsg->par = capi_cmd2par(cmsg->Command, cmsg->Subcommand);
+ if (!cmsg->par)
+ return 1; /* invalid command/subcommand */
pars_2_message(cmsg);
cmsg->p = 0;
byteTRcpy(cmsg->m + 4, &cmsg->Command);
byteTRcpy(cmsg->m + 5, &cmsg->Subcommand);
- cmsg->par = cpars[command_2_index(cmsg->Command, cmsg->Subcommand)];
+ cmsg->par = capi_cmd2par(cmsg->Command, cmsg->Subcommand);
+ if (!cmsg->par)
+ return 1; /* invalid command/subcommand */
message_2_pars(cmsg);
* @cmd: command number
* @subcmd: subcommand number
*
- * Return value: static string, NULL if command/subcommand unknown
+ * Return value: static string
*/
char *capi_cmd2str(u8 cmd, u8 subcmd)
{
- return mnames[command_2_index(cmd, subcmd)];
+ char *result;
+
+ result = mnames[command_2_index(cmd, subcmd)];
+ if (result == NULL)
+ result = "INVALID_COMMAND";
+ return result;
}
static _cdebbuf *protocol_message_2_pars(_cdebbuf *cdb, _cmsg *cmsg, int level)
{
+ if (!cmsg->par)
+ return NULL; /* invalid command/subcommand */
+
for (; TYP != _CEND; cmsg->p++) {
int slen = 29 + 3 - level;
int i;
cmsg->p = 0;
byteTRcpy(cmsg->m + 4, &cmsg->Command);
byteTRcpy(cmsg->m + 5, &cmsg->Subcommand);
- cmsg->par = cpars[command_2_index(cmsg->Command, cmsg->Subcommand)];
+ cmsg->par = capi_cmd2par(cmsg->Command, cmsg->Subcommand);
cdb = bufprint(cdb, "%-26s ID=%03d #0x%04x LEN=%04d\n",
- mnames[command_2_index(cmsg->Command, cmsg->Subcommand)],
+ capi_cmd2str(cmsg->Command, cmsg->Subcommand),
((unsigned short *) msg)[1],
((unsigned short *) msg)[3],
((unsigned short *) msg)[0]);
cmsg->l = 8;
cmsg->p = 0;
cdb = bufprint(cdb, "%s ID=%03d #0x%04x LEN=%04d\n",
- mnames[command_2_index(cmsg->Command, cmsg->Subcommand)],
+ capi_cmd2str(cmsg->Command, cmsg->Subcommand),
((u16 *) cmsg->m)[1],
((u16 *) cmsg->m)[3],
((u16 *) cmsg->m)[0]);
* Return value: CAPI result code
*/
-int capi20_manufacturer(unsigned int cmd, void __user *data)
+int capi20_manufacturer(unsigned long cmd, void __user *data)
{
struct capi_ctr *ctr;
int retval;
}
default:
- printk(KERN_ERR "kcapi: manufacturer command %d unknown.\n",
+ printk(KERN_ERR "kcapi: manufacturer command %lu unknown.\n",
cmd);
break;
l -= 12;
if (l <= 0)
return;
+ if (l > 64)
+ l = 64; /* arbitrary limit */
dbgline = kmalloc(3 * l, GFP_ATOMIC);
if (!dbgline)
return;
__func__);
break;
}
- capi_cmsg2message(&iif->hcmsg, __skb_put(skb, msgsize));
+ if (capi_cmsg2message(&iif->hcmsg,
+ __skb_put(skb, msgsize))) {
+ dev_err(cs->dev, "%s: message parser failure\n",
+ __func__);
+ dev_kfree_skb_any(skb);
+ break;
+ }
dump_cmsg(DEBUG_CMD, __func__, &iif->hcmsg);
/* add to listeners on this B channel, update state */
dev_err(cs->dev, "%s: out of memory\n", __func__);
return;
}
- capi_cmsg2message(&iif->hcmsg, __skb_put(skb, CAPI_DISCONNECT_IND_LEN));
+ if (capi_cmsg2message(&iif->hcmsg,
+ __skb_put(skb, CAPI_DISCONNECT_IND_LEN))) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, &iif->hcmsg);
capi_ctr_handle_message(&iif->ctr, ap->id, skb);
}
dev_err(cs->dev, "%s: out of memory\n", __func__);
return;
}
- capi_cmsg2message(&iif->hcmsg,
- __skb_put(skb, CAPI_DISCONNECT_B3_IND_BASELEN));
+ if (capi_cmsg2message(&iif->hcmsg,
+ __skb_put(skb, CAPI_DISCONNECT_B3_IND_BASELEN))) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, &iif->hcmsg);
capi_ctr_handle_message(&iif->ctr, ap->id, skb);
}
dev_err(cs->dev, "%s: out of memory\n", __func__);
return;
}
- capi_cmsg2message(&iif->hcmsg, __skb_put(skb, msgsize));
+ if (capi_cmsg2message(&iif->hcmsg, __skb_put(skb, msgsize))) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, &iif->hcmsg);
capi_ctr_handle_message(&iif->ctr, ap->id, skb);
}
dev_err(cs->dev, "%s: out of memory\n", __func__);
return;
}
- capi_cmsg2message(&iif->hcmsg, __skb_put(skb, msgsize));
+ if (capi_cmsg2message(&iif->hcmsg, __skb_put(skb, msgsize))) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, &iif->hcmsg);
capi_ctr_handle_message(&iif->ctr, ap->id, skb);
}
struct sk_buff *skb,
u16 info)
{
+ struct cardstate *cs = iif->ctr.driverdata;
+
/*
* _CONF replies always only have NCCI and Info parameters
* so they'll fit into the _REQ message skb
*/
capi_cmsg_answer(&iif->acmsg);
iif->acmsg.Info = info;
- capi_cmsg2message(&iif->acmsg, skb->data);
+ if (capi_cmsg2message(&iif->acmsg, skb->data)) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
__skb_trim(skb, CAPI_STDCONF_LEN);
dump_cmsg(DEBUG_CMD, __func__, &iif->acmsg);
capi_ctr_handle_message(&iif->ctr, ap->id, skb);
static u8 confparam[10]; /* max. 9 octets + length byte */
/* decode message */
- capi_message2cmsg(cmsg, skb->data);
+ if (capi_message2cmsg(cmsg, skb->data)) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, cmsg);
/*
confparam[3] = 2; /* length */
capimsg_setu16(confparam, 4,
CapiSupplementaryServiceNotSupported);
+ break;
}
info = CapiSuccess;
confparam[3] = 2; /* length */
}
/* send FACILITY_CONF with given Info and confirmation parameter */
+ dev_kfree_skb_any(skb);
capi_cmsg_answer(cmsg);
cmsg->Info = info;
cmsg->FacilityConfirmationParameter = confparam;
dev_err(cs->dev, "%s: out of memory\n", __func__);
return;
}
- capi_cmsg2message(cmsg, __skb_put(cskb, msgsize));
+ if (capi_cmsg2message(cmsg, __skb_put(cskb, msgsize))) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(cskb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, cmsg);
capi_ctr_handle_message(&iif->ctr, ap->id, cskb);
}
struct gigaset_capi_appl *ap,
struct sk_buff *skb)
{
+ struct cardstate *cs = iif->ctr.driverdata;
+
/* decode message */
- capi_message2cmsg(&iif->acmsg, skb->data);
+ if (capi_message2cmsg(&iif->acmsg, skb->data)) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, &iif->acmsg);
/* store listening parameters */
struct gigaset_capi_appl *ap,
struct sk_buff *skb)
{
+ struct cardstate *cs = iif->ctr.driverdata;
+
/* decode message */
- capi_message2cmsg(&iif->acmsg, skb->data);
+ if (capi_message2cmsg(&iif->acmsg, skb->data)) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, &iif->acmsg);
send_conf(iif, ap, skb, CapiAlertAlreadySent);
}
u16 info;
/* decode message */
- capi_message2cmsg(cmsg, skb->data);
+ if (capi_message2cmsg(cmsg, skb->data)) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, cmsg);
/* get free B channel & construct PLCI */
int channel;
/* decode message */
- capi_message2cmsg(cmsg, skb->data);
+ if (capi_message2cmsg(cmsg, skb->data)) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, cmsg);
dev_kfree_skb_any(skb);
int channel;
/* decode message */
- capi_message2cmsg(cmsg, skb->data);
+ if (capi_message2cmsg(cmsg, skb->data)) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, cmsg);
/* extract and check channel number from PLCI */
u8 command;
/* decode message */
- capi_message2cmsg(cmsg, skb->data);
+ if (capi_message2cmsg(cmsg, skb->data)) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, cmsg);
/* extract and check channel number and NCCI */
capi_cmsg_header(cmsg, ap->id, command, CAPI_IND,
ap->nextMessageNumber++, cmsg->adr.adrNCCI);
__skb_trim(skb, msgsize);
- capi_cmsg2message(cmsg, skb->data);
+ if (capi_cmsg2message(cmsg, skb->data)) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, cmsg);
capi_ctr_handle_message(&iif->ctr, ap->id, skb);
}
int channel;
/* decode message */
- capi_message2cmsg(cmsg, skb->data);
+ if (capi_message2cmsg(cmsg, skb->data)) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, cmsg);
/* extract and check channel number from PLCI */
kfree(b3cmsg);
return;
}
- capi_cmsg2message(b3cmsg,
- __skb_put(b3skb, CAPI_DISCONNECT_B3_IND_BASELEN));
+ if (capi_cmsg2message(b3cmsg,
+ __skb_put(b3skb, CAPI_DISCONNECT_B3_IND_BASELEN))) {
+ dev_err(cs->dev, "%s: message parser failure\n",
+ __func__);
+ kfree(b3cmsg);
+ dev_kfree_skb_any(b3skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, b3cmsg);
kfree(b3cmsg);
capi_ctr_handle_message(&iif->ctr, ap->id, b3skb);
int channel;
/* decode message */
- capi_message2cmsg(cmsg, skb->data);
+ if (capi_message2cmsg(cmsg, skb->data)) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, cmsg);
/* extract and check channel number and NCCI */
struct gigaset_capi_appl *ap,
struct sk_buff *skb)
{
+ struct cardstate *cs = iif->ctr.driverdata;
+
/* decode message */
- capi_message2cmsg(&iif->acmsg, skb->data);
+ if (capi_message2cmsg(&iif->acmsg, skb->data)) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, &iif->acmsg);
send_conf(iif, ap, skb,
CapiResetProcedureNotSupportedByCurrentProtocol);
struct gigaset_capi_appl *ap,
struct sk_buff *skb)
{
+ struct cardstate *cs = iif->ctr.driverdata;
+
/* decode message */
- capi_message2cmsg(&iif->acmsg, skb->data);
+ if (capi_message2cmsg(&iif->acmsg, skb->data)) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, &iif->acmsg);
send_conf(iif, ap, skb, CapiMessageNotSupportedInCurrentState);
}
struct gigaset_capi_appl *ap,
struct sk_buff *skb)
{
+ struct cardstate *cs = iif->ctr.driverdata;
+
/* decode message */
- capi_message2cmsg(&iif->acmsg, skb->data);
+ if (capi_message2cmsg(&iif->acmsg, skb->data)) {
+ dev_err(cs->dev, "%s: message parser failure\n", __func__);
+ dev_kfree_skb_any(skb);
+ return;
+ }
dump_cmsg(DEBUG_CMD, __func__, &iif->acmsg);
dev_kfree_skb_any(skb);
}
struct gigaset_capi_ctr *iif;
int rc;
- iif = kmalloc(sizeof(*iif), GFP_KERNEL);
+ iif = kzalloc(sizeof(*iif), GFP_KERNEL);
if (!iif) {
pr_err("%s: out of memory\n", __func__);
return -ENOMEM;
/* prepare controller structure */
iif->ctr.owner = THIS_MODULE;
iif->ctr.driverdata = cs;
- strncpy(iif->ctr.name, isdnid, sizeof(iif->ctr.name));
+ strncpy(iif->ctr.name, isdnid, sizeof(iif->ctr.name) - 1);
iif->ctr.driver_name = "gigaset";
iif->ctr.load_firmware = NULL;
iif->ctr.reset_ctr = NULL;
}
EXPORT_SYMBOL_GPL(gigaset_handle_modem_response);
-/* disconnect
+/* disconnect_nobc
* process closing of connection associated with given AT state structure
+ * without B channel
*/
-static void disconnect(struct at_state_t **at_state_p)
+static void disconnect_nobc(struct at_state_t **at_state_p,
+ struct cardstate *cs)
{
unsigned long flags;
- struct bc_state *bcs = (*at_state_p)->bcs;
- struct cardstate *cs = (*at_state_p)->cs;
spin_lock_irqsave(&cs->lock, flags);
++(*at_state_p)->seq_index;
gig_dbg(DEBUG_EVENT, "Scheduling PC_UMMODE");
cs->commands_pending = 1;
}
- spin_unlock_irqrestore(&cs->lock, flags);
- if (bcs) {
- /* B channel assigned: invoke hardware specific handler */
- cs->ops->close_bchannel(bcs);
- /* notify LL */
- if (bcs->chstate & (CHS_D_UP | CHS_NOTIFY_LL)) {
- bcs->chstate &= ~(CHS_D_UP | CHS_NOTIFY_LL);
- gigaset_isdn_hupD(bcs);
- }
- } else {
- /* no B channel assigned: just deallocate */
- spin_lock_irqsave(&cs->lock, flags);
+ /* check for and deallocate temporary AT state */
+ if (!list_empty(&(*at_state_p)->list)) {
list_del(&(*at_state_p)->list);
kfree(*at_state_p);
*at_state_p = NULL;
- spin_unlock_irqrestore(&cs->lock, flags);
+ }
+
+ spin_unlock_irqrestore(&cs->lock, flags);
+}
+
+/* disconnect_bc
+ * process closing of connection associated with given AT state structure
+ * and B channel
+ */
+static void disconnect_bc(struct at_state_t *at_state,
+ struct cardstate *cs, struct bc_state *bcs)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&cs->lock, flags);
+ ++at_state->seq_index;
+
+ /* revert to selected idle mode */
+ if (!cs->cidmode) {
+ cs->at_state.pending_commands |= PC_UMMODE;
+ gig_dbg(DEBUG_EVENT, "Scheduling PC_UMMODE");
+ cs->commands_pending = 1;
+ }
+ spin_unlock_irqrestore(&cs->lock, flags);
+
+ /* invoke hardware specific handler */
+ cs->ops->close_bchannel(bcs);
+
+ /* notify LL */
+ if (bcs->chstate & (CHS_D_UP | CHS_NOTIFY_LL)) {
+ bcs->chstate &= ~(CHS_D_UP | CHS_NOTIFY_LL);
+ gigaset_isdn_hupD(bcs);
}
}
* get a free AT state structure: either one of those associated with the
* B channels of the Gigaset device, or if none of those is available,
* a newly allocated one with bcs=NULL
- * The structure should be freed by calling disconnect() after use.
+ * The structure should be freed by calling disconnect_nobc() after use.
*/
static inline struct at_state_t *get_free_channel(struct cardstate *cs,
int cid)
struct event_t *ev)
{
struct at_state_t *at_state = *p_at_state;
- struct at_state_t *at_state2;
+ struct bc_state *bcs2;
unsigned long flags;
int channel;
break;
case ACT_RING:
/* get fresh AT state structure for new CID */
- at_state2 = get_free_channel(cs, ev->parameter);
- if (!at_state2) {
+ at_state = get_free_channel(cs, ev->parameter);
+ if (!at_state) {
dev_warn(cs->dev,
"RING ignored: could not allocate channel structure\n");
break;
/* initialize AT state structure
* note that bcs may be NULL if no B channel is free
*/
- at_state2->ConState = 700;
+ at_state->ConState = 700;
for (i = 0; i < STR_NUM; ++i) {
- kfree(at_state2->str_var[i]);
- at_state2->str_var[i] = NULL;
+ kfree(at_state->str_var[i]);
+ at_state->str_var[i] = NULL;
}
- at_state2->int_var[VAR_ZCTP] = -1;
+ at_state->int_var[VAR_ZCTP] = -1;
spin_lock_irqsave(&cs->lock, flags);
- at_state2->timer_expires = RING_TIMEOUT;
- at_state2->timer_active = 1;
+ at_state->timer_expires = RING_TIMEOUT;
+ at_state->timer_active = 1;
spin_unlock_irqrestore(&cs->lock, flags);
break;
case ACT_ICALL:
case ACT_DISCONNECT:
cs->cur_at_seq = SEQ_NONE;
at_state->cid = -1;
- if (bcs && cs->onechannel && cs->dle) {
+ if (!bcs) {
+ disconnect_nobc(p_at_state, cs);
+ } else if (cs->onechannel && cs->dle) {
/* Check for other open channels not needed:
* DLE only used for M10x with one B channel.
*/
at_state->pending_commands |= PC_DLE0;
cs->commands_pending = 1;
- } else
- disconnect(p_at_state);
+ } else {
+ disconnect_bc(at_state, cs, bcs);
+ }
break;
case ACT_FAKEDLE0:
at_state->int_var[VAR_ZDLE] = 0;
/* fall through */
case ACT_DLE0:
cs->cur_at_seq = SEQ_NONE;
- at_state2 = &cs->bcs[cs->curchannel].at_state;
- disconnect(&at_state2);
+ bcs2 = cs->bcs + cs->curchannel;
+ disconnect_bc(&bcs2->at_state, cs, bcs2);
break;
case ACT_ABORTHUP:
cs->cur_at_seq = SEQ_NONE;
dev_warn(cs->dev, "Could not hang up.\n");
at_state->cid = -1;
- if (bcs && cs->onechannel)
+ if (!bcs)
+ disconnect_nobc(p_at_state, cs);
+ else if (cs->onechannel)
at_state->pending_commands |= PC_DLE0;
else
- disconnect(p_at_state);
+ disconnect_bc(at_state, cs, bcs);
schedule_init(cs, MS_RECOVER);
break;
case ACT_FAILDLE0:
cs->cur_at_seq = SEQ_NONE;
dev_warn(cs->dev, "Error leaving DLE mode.\n");
cs->dle = 0;
- at_state2 = &cs->bcs[cs->curchannel].at_state;
- disconnect(&at_state2);
+ bcs2 = cs->bcs + cs->curchannel;
+ disconnect_bc(&bcs2->at_state, cs, bcs2);
schedule_init(cs, MS_RECOVER);
break;
case ACT_FAILDLE1:
if (reinit_and_retry(cs, channel) < 0) {
dev_warn(cs->dev,
"Could not get a call ID. Cannot dial.\n");
- at_state2 = &cs->bcs[channel].at_state;
- disconnect(&at_state2);
+ bcs2 = cs->bcs + channel;
+ disconnect_bc(&bcs2->at_state, cs, bcs2);
}
break;
case ACT_ABORTCID:
cs->cur_at_seq = SEQ_NONE;
- at_state2 = &cs->bcs[cs->curchannel].at_state;
- disconnect(&at_state2);
+ bcs2 = cs->bcs + cs->curchannel;
+ disconnect_bc(&bcs2->at_state, cs, bcs2);
break;
case ACT_DIALING:
break;
case ACT_ABORTACCEPT: /* hangup/error/timeout during ICALL procssng */
- disconnect(p_at_state);
+ if (bcs)
+ disconnect_bc(at_state, cs, bcs);
+ else
+ disconnect_nobc(p_at_state, cs);
break;
case ACT_ABORTDIAL: /* error/timeout during dial preparation */
/* events from the LL */
case ACT_DIAL:
+ if (!ev->ptr) {
+ *p_genresp = 1;
+ *p_resp_code = RSP_ERROR;
+ break;
+ }
start_dial(at_state, ev->ptr, ev->parameter);
break;
case ACT_ACCEPT:
static int gigaset_write_cmd(struct cardstate *cs, struct cmdbuf_t *cb)
{
unsigned long flags;
+ int len;
gigaset_dbg_buffer(cs->mstate != MS_LOCKED ?
DEBUG_TRANSCMD : DEBUG_LOCKCMD,
spin_unlock_irqrestore(&cs->cmdlock, flags);
spin_lock_irqsave(&cs->lock, flags);
+ len = cb->len;
if (cs->connected)
tasklet_schedule(&cs->write_tasklet);
spin_unlock_irqrestore(&cs->lock, flags);
- return cb->len;
+ return len;
}
static int gigaset_write_room(struct cardstate *cs)
q->limits.logical_block_size = block_size;
q->limits.physical_block_size = block_size;
set_bit(QUEUE_FLAG_NONROT, &d->disk->queue->queue_flags);
+ clear_bit(QUEUE_FLAG_ADD_RANDOM, &d->disk->queue->queue_flags);
set_bit(QUEUE_FLAG_DISCARD, &d->disk->queue->queue_flags);
blk_queue_flush(q, REQ_FLUSH|REQ_FUA);
{
unsigned long i;
int dirty, need_write;
- int wait = 0;
if (!bitmap || !bitmap->storage.filemap ||
test_bit(BITMAP_STALE, &bitmap->flags))
clear_page_attr(bitmap, i, BITMAP_PAGE_PENDING);
write_page(bitmap, bitmap->storage.filemap[i], 0);
}
- if (dirty)
- wait = 1;
- }
- if (wait) { /* if any writes were performed, we need to wait on them */
- if (bitmap->storage.file)
- wait_event(bitmap->write_wait,
- atomic_read(&bitmap->pending_writes)==0);
- else
- md_super_wait(bitmap->mddev);
}
+ if (bitmap->storage.file)
+ wait_event(bitmap->write_wait,
+ atomic_read(&bitmap->pending_writes)==0);
+ else
+ md_super_wait(bitmap->mddev);
+
if (test_bit(BITMAP_WRITE_ERROR, &bitmap->flags))
bitmap_file_kick(bitmap);
}
c->n_buffers[dirty]++;
b->list_mode = dirty;
list_move(&b->lru_list, &c->lru[dirty]);
+ b->last_accessed = jiffies;
}
/*----------------------------------------------------------------
list_for_each_entry_safe_reverse(b, tmp, &c->lru[l], lru_list) {
freed += __cleanup_old_buffer(b, gfp_mask, 0);
if (!--nr_to_scan)
- break;
+ return freed;
+ dm_bufio_cond_resched();
}
- dm_bufio_cond_resched();
}
return freed;
}
deps->count = count;
count = 0;
list_for_each_entry (dd, dm_table_get_devices(table), list)
- deps->dev[count++] = huge_encode_dev(dd->dm_dev.bdev->bd_dev);
+ deps->dev[count++] = huge_encode_dev(dd->dm_dev->bdev->bd_dev);
param->data_size = param->data_start + needed;
}
r = cn_add_callback(&ulog_cn_id, "dmlogusr", cn_ulog_callback);
if (r) {
- cn_del_callback(&ulog_cn_id);
+ kfree(prealloced_cn_msg);
return r;
}
struct priority_group *pg;
unsigned bypassed = 1;
- if (!m->nr_valid_paths)
+ if (!m->nr_valid_paths) {
+ m->queue_io = 0;
goto failed;
+ }
/* Were we instructed to switch PG? */
if (m->next_pg) {
/*
* Copyright (C) 2010-2011 Neil Brown
- * Copyright (C) 2010-2011 Red Hat, Inc. All rights reserved.
+ * Copyright (C) 2010-2014 Red Hat, Inc. All rights reserved.
*
* This file is released under the GPL.
*/
#define DM_MSG_PREFIX "raid"
+static bool devices_handle_discard_safely = false;
+
/*
* The following flags are used by dm-raid.c to set up the array state.
* They must be cleared before md_run is called.
* will form the "stripe"
* [[no]sync] Force or prevent recovery of the
* entire array
+ * [devices_handle_discard_safely] Allow discards on RAID4/5/6; useful if RAID
+ * member device(s) properly support TRIM/UNMAP
* [rebuild <idx>] Rebuild the drive indicated by the index
* [daemon_sleep <ms>] Time between bitmap daemon work to
* clear bits
return 0;
}
+/*
+ * Enable/disable discard support on RAID set depending on
+ * RAID level and discard properties of underlying RAID members.
+ */
+static void configure_discard_support(struct dm_target *ti, struct raid_set *rs)
+{
+ int i;
+ bool raid456;
+
+ /* Assume discards not supported until after checks below. */
+ ti->discards_supported = false;
+
+ /* RAID level 4,5,6 require discard_zeroes_data for data integrity! */
+ raid456 = (rs->md.level == 4 || rs->md.level == 5 || rs->md.level == 6);
+
+ for (i = 0; i < rs->md.raid_disks; i++) {
+ struct request_queue *q = bdev_get_queue(rs->dev[i].rdev.bdev);
+
+ if (!q || !blk_queue_discard(q))
+ return;
+
+ if (raid456) {
+ if (!q->limits.discard_zeroes_data)
+ return;
+ if (!devices_handle_discard_safely) {
+ DMERR("raid456 discard support disabled due to discard_zeroes_data uncertainty.");
+ DMERR("Set dm-raid.devices_handle_discard_safely=Y to override.");
+ return;
+ }
+ }
+ }
+
+ /* All RAID members properly support discards */
+ ti->discards_supported = true;
+
+ /*
+ * RAID1 and RAID10 personalities require bio splitting,
+ * RAID0/4/5/6 don't and process large discard bios properly.
+ */
+ ti->split_discard_bios = !!(rs->md.level == 1 || rs->md.level == 10);
+ ti->num_discard_bios = 1;
+}
+
/*
* Construct a RAID4/5/6 mapping:
* Args:
ti->private = rs;
ti->num_flush_bios = 1;
+ /*
+ * Disable/enable discard support on RAID set.
+ */
+ configure_discard_support(ti, rs);
+
mutex_lock(&rs->md.reconfig_mutex);
ret = md_run(&rs->md);
rs->md.in_sync = 0; /* Assume already marked dirty */
static struct target_type raid_target = {
.name = "raid",
- .version = {1, 5, 2},
+ .version = {1, 6, 0},
.module = THIS_MODULE,
.ctr = raid_ctr,
.dtr = raid_dtr,
module_init(dm_raid_init);
module_exit(dm_raid_exit);
+module_param(devices_handle_discard_safely, bool, 0644);
+MODULE_PARM_DESC(devices_handle_discard_safely,
+ "Set to Y if all devices in each array reliably return zeroes on reads from discarded regions");
+
MODULE_DESCRIPTION(DM_NAME " raid4/5/6 target");
MODULE_ALIAS("dm-raid1");
MODULE_ALIAS("dm-raid10");
return 0;
}
-static void free_devices(struct list_head *devices)
+static void free_devices(struct list_head *devices, struct mapped_device *md)
{
struct list_head *tmp, *next;
list_for_each_safe(tmp, next, devices) {
struct dm_dev_internal *dd =
list_entry(tmp, struct dm_dev_internal, list);
- DMWARN("dm_table_destroy: dm_put_device call missing for %s",
- dd->dm_dev.name);
+ DMWARN("%s: dm_table_destroy: dm_put_device call missing for %s",
+ dm_device_name(md), dd->dm_dev->name);
+ dm_put_table_device(md, dd->dm_dev);
kfree(dd);
}
}
vfree(t->highs);
/* free the device list */
- free_devices(&t->devices);
+ free_devices(&t->devices, t->md);
dm_free_md_mempools(t->mempools);
struct dm_dev_internal *dd;
list_for_each_entry (dd, l, list)
- if (dd->dm_dev.bdev->bd_dev == dev)
+ if (dd->dm_dev->bdev->bd_dev == dev)
return dd;
return NULL;
}
-/*
- * Open a device so we can use it as a map destination.
- */
-static int open_dev(struct dm_dev_internal *d, dev_t dev,
- struct mapped_device *md)
-{
- static char *_claim_ptr = "I belong to device-mapper";
- struct block_device *bdev;
-
- int r;
-
- BUG_ON(d->dm_dev.bdev);
-
- bdev = blkdev_get_by_dev(dev, d->dm_dev.mode | FMODE_EXCL, _claim_ptr);
- if (IS_ERR(bdev))
- return PTR_ERR(bdev);
-
- r = bd_link_disk_holder(bdev, dm_disk(md));
- if (r) {
- blkdev_put(bdev, d->dm_dev.mode | FMODE_EXCL);
- return r;
- }
-
- d->dm_dev.bdev = bdev;
- return 0;
-}
-
-/*
- * Close a device that we've been using.
- */
-static void close_dev(struct dm_dev_internal *d, struct mapped_device *md)
-{
- if (!d->dm_dev.bdev)
- return;
-
- bd_unlink_disk_holder(d->dm_dev.bdev, dm_disk(md));
- blkdev_put(d->dm_dev.bdev, d->dm_dev.mode | FMODE_EXCL);
- d->dm_dev.bdev = NULL;
-}
-
/*
* If possible, this checks an area of a destination device is invalid.
*/
struct mapped_device *md)
{
int r;
- struct dm_dev_internal dd_new, dd_old;
+ struct dm_dev *old_dev, *new_dev;
- dd_new = dd_old = *dd;
+ old_dev = dd->dm_dev;
- dd_new.dm_dev.mode |= new_mode;
- dd_new.dm_dev.bdev = NULL;
-
- r = open_dev(&dd_new, dd->dm_dev.bdev->bd_dev, md);
+ r = dm_get_table_device(md, dd->dm_dev->bdev->bd_dev,
+ dd->dm_dev->mode | new_mode, &new_dev);
if (r)
return r;
- dd->dm_dev.mode |= new_mode;
- close_dev(&dd_old, md);
+ dd->dm_dev = new_dev;
+ dm_put_table_device(md, old_dev);
return 0;
}
if (!dd)
return -ENOMEM;
- dd->dm_dev.mode = mode;
- dd->dm_dev.bdev = NULL;
-
- if ((r = open_dev(dd, dev, t->md))) {
+ if ((r = dm_get_table_device(t->md, dev, mode, &dd->dm_dev))) {
kfree(dd);
return r;
}
- format_dev_t(dd->dm_dev.name, dev);
-
atomic_set(&dd->count, 0);
list_add(&dd->list, &t->devices);
- } else if (dd->dm_dev.mode != (mode | dd->dm_dev.mode)) {
+ } else if (dd->dm_dev->mode != (mode | dd->dm_dev->mode)) {
r = upgrade_mode(dd, mode, t->md);
if (r)
return r;
}
atomic_inc(&dd->count);
- *result = &dd->dm_dev;
+ *result = dd->dm_dev;
return 0;
}
EXPORT_SYMBOL(dm_get_device);
*/
void dm_put_device(struct dm_target *ti, struct dm_dev *d)
{
- struct dm_dev_internal *dd = container_of(d, struct dm_dev_internal,
- dm_dev);
+ int found = 0;
+ struct list_head *devices = &ti->table->devices;
+ struct dm_dev_internal *dd;
+ list_for_each_entry(dd, devices, list) {
+ if (dd->dm_dev == d) {
+ found = 1;
+ break;
+ }
+ }
+ if (!found) {
+ DMWARN("%s: device %s not in table devices list",
+ dm_device_name(ti->table->md), d->name);
+ return;
+ }
if (atomic_dec_and_test(&dd->count)) {
- close_dev(dd, ti->table->md);
+ dm_put_table_device(ti->table->md, d);
list_del(&dd->list);
kfree(dd);
}
/* Non-request-stackable devices can't be used for request-based dm */
devices = dm_table_get_devices(t);
list_for_each_entry(dd, devices, list) {
- if (!blk_queue_stackable(bdev_get_queue(dd->dm_dev.bdev))) {
+ if (!blk_queue_stackable(bdev_get_queue(dd->dm_dev->bdev))) {
DMWARN("table load rejected: including"
" non-request-stackable devices");
return -EINVAL;
struct gendisk *prev_disk = NULL, *template_disk = NULL;
list_for_each_entry(dd, devices, list) {
- template_disk = dd->dm_dev.bdev->bd_disk;
+ template_disk = dd->dm_dev->bdev->bd_disk;
if (!blk_get_integrity(template_disk))
goto no_integrity;
if (!match_all && !blk_integrity_is_initialized(template_disk))
int r = 0;
list_for_each_entry(dd, devices, list) {
- struct request_queue *q = bdev_get_queue(dd->dm_dev.bdev);
+ struct request_queue *q = bdev_get_queue(dd->dm_dev->bdev);
char b[BDEVNAME_SIZE];
if (likely(q))
else
DMWARN_LIMIT("%s: any_congested: nonexistent device %s",
dm_device_name(t->md),
- bdevname(dd->dm_dev.bdev, b));
+ bdevname(dd->dm_dev->bdev, b));
}
list_for_each_entry(cb, &t->target_callbacks, list)
*/
struct dm_table *map;
+ struct list_head table_devices;
+ struct mutex table_devices_lock;
+
unsigned long flags;
struct request_queue *queue;
struct bio_set *bs;
};
+struct table_device {
+ struct list_head list;
+ atomic_t count;
+ struct dm_dev dm_dev;
+};
+
#define RESERVED_BIO_BASED_IOS 16
#define RESERVED_REQUEST_BASED_IOS 256
#define RESERVED_MAX_IOS 1024
rcu_read_unlock();
}
+/*
+ * Open a table device so we can use it as a map destination.
+ */
+static int open_table_device(struct table_device *td, dev_t dev,
+ struct mapped_device *md)
+{
+ static char *_claim_ptr = "I belong to device-mapper";
+ struct block_device *bdev;
+
+ int r;
+
+ BUG_ON(td->dm_dev.bdev);
+
+ bdev = blkdev_get_by_dev(dev, td->dm_dev.mode | FMODE_EXCL, _claim_ptr);
+ if (IS_ERR(bdev))
+ return PTR_ERR(bdev);
+
+ r = bd_link_disk_holder(bdev, dm_disk(md));
+ if (r) {
+ blkdev_put(bdev, td->dm_dev.mode | FMODE_EXCL);
+ return r;
+ }
+
+ td->dm_dev.bdev = bdev;
+ return 0;
+}
+
+/*
+ * Close a table device that we've been using.
+ */
+static void close_table_device(struct table_device *td, struct mapped_device *md)
+{
+ if (!td->dm_dev.bdev)
+ return;
+
+ bd_unlink_disk_holder(td->dm_dev.bdev, dm_disk(md));
+ blkdev_put(td->dm_dev.bdev, td->dm_dev.mode | FMODE_EXCL);
+ td->dm_dev.bdev = NULL;
+}
+
+static struct table_device *find_table_device(struct list_head *l, dev_t dev,
+ fmode_t mode) {
+ struct table_device *td;
+
+ list_for_each_entry(td, l, list)
+ if (td->dm_dev.bdev->bd_dev == dev && td->dm_dev.mode == mode)
+ return td;
+
+ return NULL;
+}
+
+int dm_get_table_device(struct mapped_device *md, dev_t dev, fmode_t mode,
+ struct dm_dev **result) {
+ int r;
+ struct table_device *td;
+
+ mutex_lock(&md->table_devices_lock);
+ td = find_table_device(&md->table_devices, dev, mode);
+ if (!td) {
+ td = kmalloc(sizeof(*td), GFP_KERNEL);
+ if (!td) {
+ mutex_unlock(&md->table_devices_lock);
+ return -ENOMEM;
+ }
+
+ td->dm_dev.mode = mode;
+ td->dm_dev.bdev = NULL;
+
+ if ((r = open_table_device(td, dev, md))) {
+ mutex_unlock(&md->table_devices_lock);
+ kfree(td);
+ return r;
+ }
+
+ format_dev_t(td->dm_dev.name, dev);
+
+ atomic_set(&td->count, 0);
+ list_add(&td->list, &md->table_devices);
+ }
+ atomic_inc(&td->count);
+ mutex_unlock(&md->table_devices_lock);
+
+ *result = &td->dm_dev;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(dm_get_table_device);
+
+void dm_put_table_device(struct mapped_device *md, struct dm_dev *d)
+{
+ struct table_device *td = container_of(d, struct table_device, dm_dev);
+
+ mutex_lock(&md->table_devices_lock);
+ if (atomic_dec_and_test(&td->count)) {
+ close_table_device(td, md);
+ list_del(&td->list);
+ kfree(td);
+ }
+ mutex_unlock(&md->table_devices_lock);
+}
+EXPORT_SYMBOL(dm_put_table_device);
+
+static void free_table_devices(struct list_head *devices)
+{
+ struct list_head *tmp, *next;
+
+ list_for_each_safe(tmp, next, devices) {
+ struct table_device *td = list_entry(tmp, struct table_device, list);
+
+ DMWARN("dm_destroy: %s still exists with %d references",
+ td->dm_dev.name, atomic_read(&td->count));
+ kfree(td);
+ }
+}
+
/*
* Get the geometry associated with a dm device
*/
}
static struct dm_target_io *alloc_tio(struct clone_info *ci,
- struct dm_target *ti, int nr_iovecs,
+ struct dm_target *ti,
unsigned target_bio_nr)
{
struct dm_target_io *tio;
struct bio *clone;
- clone = bio_alloc_bioset(GFP_NOIO, nr_iovecs, ci->md->bs);
+ clone = bio_alloc_bioset(GFP_NOIO, 0, ci->md->bs);
tio = container_of(clone, struct dm_target_io, clone);
tio->io = ci->io;
struct dm_target *ti,
unsigned target_bio_nr, unsigned *len)
{
- struct dm_target_io *tio = alloc_tio(ci, ti, ci->bio->bi_max_vecs, target_bio_nr);
+ struct dm_target_io *tio = alloc_tio(ci, ti, target_bio_nr);
struct bio *clone = &tio->clone;
tio->len_ptr = len;
- /*
- * Discard requests require the bio's inline iovecs be initialized.
- * ci->bio->bi_max_vecs is BIO_INLINE_VECS anyway, for both flush
- * and discard, so no need for concern about wasted bvec allocations.
- */
- __bio_clone_fast(clone, ci->bio);
+ __bio_clone_fast(clone, ci->bio);
if (len)
bio_setup_sector(clone, ci->sector, *len);
num_target_bios = ti->num_write_bios(ti, bio);
for (target_bio_nr = 0; target_bio_nr < num_target_bios; target_bio_nr++) {
- tio = alloc_tio(ci, ti, 0, target_bio_nr);
+ tio = alloc_tio(ci, ti, target_bio_nr);
tio->len_ptr = len;
clone_bio(tio, bio, sector, *len);
__map_bio(tio);
md->type = DM_TYPE_NONE;
mutex_init(&md->suspend_lock);
mutex_init(&md->type_lock);
+ mutex_init(&md->table_devices_lock);
spin_lock_init(&md->deferred_lock);
atomic_set(&md->holders, 1);
atomic_set(&md->open_count, 0);
atomic_set(&md->event_nr, 0);
atomic_set(&md->uevent_seq, 0);
INIT_LIST_HEAD(&md->uevent_list);
+ INIT_LIST_HEAD(&md->table_devices);
spin_lock_init(&md->uevent_lock);
md->queue = blk_alloc_queue(GFP_KERNEL);
blk_integrity_unregister(md->disk);
del_gendisk(md->disk);
cleanup_srcu_struct(&md->io_barrier);
+ free_table_devices(&md->table_devices);
free_minor(minor);
spin_lock(&_minor_lock);
if (!pools->io_pool)
goto out;
- pools->bs = bioset_create(pool_size, front_pad);
+ pools->bs = bioset_create_nobvec(pool_size, front_pad);
if (!pools->bs)
goto out;
struct dm_dev_internal {
struct list_head list;
atomic_t count;
- struct dm_dev dm_dev;
+ struct dm_dev *dm_dev;
};
struct dm_table;
int dm_request_based(struct mapped_device *md);
sector_t dm_get_size(struct mapped_device *md);
struct request_queue *dm_get_md_queue(struct mapped_device *md);
+int dm_get_table_device(struct mapped_device *md, dev_t dev, fmode_t mode,
+ struct dm_dev **result);
+void dm_put_table_device(struct mapped_device *md, struct dm_dev *d);
struct dm_stats *dm_get_stats(struct mapped_device *md);
int dm_kobject_uevent(struct mapped_device *md, enum kobject_action action,
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2, or (at your option)
any later version.
-
+
You should have received a copy of the GNU General Public License
(for example /usr/src/linux/COPYING); if not, write to the Free
- Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#include <linux/blkdev.h>
#include "linear.h"
/*
- * find which device holds a particular offset
+ * find which device holds a particular offset
*/
static inline struct dev_info *which_dev(struct mddev *mddev, sector_t sector)
{
seq_printf(seq, " %dk rounding", mddev->chunk_sectors / 2);
}
-
static struct md_personality linear_personality =
{
.name = "linear",
unregister_md_personality (&linear_personality);
}
-
module_init(linear_init);
module_exit(linear_exit);
MODULE_LICENSE("GPL");
/*
md.c : Multiple Devices driver for Linux
- Copyright (C) 1998, 1999, 2000 Ingo Molnar
+ Copyright (C) 1998, 1999, 2000 Ingo Molnar
completely rewritten, based on the MD driver code from Marc Zyngier
static LIST_HEAD(pers_list);
static DEFINE_SPINLOCK(pers_lock);
-static void md_print_devices(void);
-
static DECLARE_WAIT_QUEUE_HEAD(resync_wait);
static struct workqueue_struct *md_wq;
static struct workqueue_struct *md_misc_wq;
static int remove_and_add_spares(struct mddev *mddev,
struct md_rdev *this);
-#define MD_BUG(x...) { printk("md: bug in file %s, line %d\n", __FILE__, __LINE__); md_print_devices(); }
-
/*
* Default number of read corrections we'll attempt on an rdev
* before ejecting it from the array. We divide the read error
static LIST_HEAD(all_mddevs);
static DEFINE_SPINLOCK(all_mddevs_lock);
-
/*
* iterates through all used mddevs in the system.
* We take care to grab the all_mddevs_lock whenever navigating
*/
#define for_each_mddev(_mddev,_tmp) \
\
- for (({ spin_lock(&all_mddevs_lock); \
+ for (({ spin_lock(&all_mddevs_lock); \
_tmp = all_mddevs.next; \
_mddev = NULL;}); \
({ if (_tmp != &all_mddevs) \
_tmp = _tmp->next;}) \
)
-
/* Rather than calling directly into the personality make_request function,
* IO requests come here first so that we can check if the device is
* being suspended pending a reconfiguration.
}
EXPORT_SYMBOL_GPL(mddev_init);
-static struct mddev * mddev_find(dev_t unit)
+static struct mddev *mddev_find(dev_t unit)
{
struct mddev *mddev, *new = NULL;
kfree(new);
return NULL;
}
-
+
is_free = 1;
list_for_each_entry(mddev, &all_mddevs, all_mddevs)
if (mddev->unit == dev) {
goto retry;
}
-static inline int __must_check mddev_lock(struct mddev * mddev)
+static inline int __must_check mddev_lock(struct mddev *mddev)
{
return mutex_lock_interruptible(&mddev->reconfig_mutex);
}
/* Sometimes we need to take the lock in a situation where
* failure due to interrupts is not acceptable.
*/
-static inline void mddev_lock_nointr(struct mddev * mddev)
+static inline void mddev_lock_nointr(struct mddev *mddev)
{
mutex_lock(&mddev->reconfig_mutex);
}
return mutex_is_locked(&mddev->reconfig_mutex);
}
-static inline int mddev_trylock(struct mddev * mddev)
+static inline int mddev_trylock(struct mddev *mddev)
{
return mutex_trylock(&mddev->reconfig_mutex);
}
static struct attribute_group md_redundancy_group;
-static void mddev_unlock(struct mddev * mddev)
+static void mddev_unlock(struct mddev *mddev)
{
if (mddev->to_remove) {
/* These cannot be removed under reconfig_mutex as
spin_unlock(&pers_lock);
}
-static struct md_rdev * find_rdev_nr(struct mddev *mddev, int nr)
-{
- struct md_rdev *rdev;
-
- rdev_for_each(rdev, mddev)
- if (rdev->desc_nr == nr)
- return rdev;
-
- return NULL;
-}
-
static struct md_rdev *find_rdev_nr_rcu(struct mddev *mddev, int nr)
{
struct md_rdev *rdev;
return MD_NEW_SIZE_SECTORS(num_sectors);
}
-static int alloc_disk_sb(struct md_rdev * rdev)
+static int alloc_disk_sb(struct md_rdev *rdev)
{
- if (rdev->sb_page)
- MD_BUG();
-
rdev->sb_page = alloc_page(GFP_KERNEL);
if (!rdev->sb_page) {
printk(KERN_ALERT "md: out of memory.\n");
void md_super_wait(struct mddev *mddev)
{
/* wait for all superblock writes that were scheduled to complete */
- DEFINE_WAIT(wq);
- for(;;) {
- prepare_to_wait(&mddev->sb_wait, &wq, TASK_UNINTERRUPTIBLE);
- if (atomic_read(&mddev->pending_writes)==0)
- break;
- schedule();
- }
- finish_wait(&mddev->sb_wait, &wq);
+ wait_event(mddev->sb_wait, atomic_read(&mddev->pending_writes)==0);
}
int sync_page_io(struct md_rdev *rdev, sector_t sector, int size,
}
EXPORT_SYMBOL_GPL(sync_page_io);
-static int read_disk_sb(struct md_rdev * rdev, int size)
+static int read_disk_sb(struct md_rdev *rdev, int size)
{
char b[BDEVNAME_SIZE];
- if (!rdev->sb_page) {
- MD_BUG();
- return -EINVAL;
- }
+
if (rdev->sb_loaded)
return 0;
-
if (!sync_page_io(rdev, 0, size, rdev->sb_page, READ, true))
goto fail;
rdev->sb_loaded = 1;
static int uuid_equal(mdp_super_t *sb1, mdp_super_t *sb2)
{
- return sb1->set_uuid0 == sb2->set_uuid0 &&
+ return sb1->set_uuid0 == sb2->set_uuid0 &&
sb1->set_uuid1 == sb2->set_uuid1 &&
sb1->set_uuid2 == sb2->set_uuid2 &&
sb1->set_uuid3 == sb2->set_uuid3;
return ret;
}
-
static u32 md_csum_fold(u32 csum)
{
csum = (csum & 0xffff) + (csum >> 16);
return (csum & 0xffff) + (csum >> 16);
}
-static unsigned int calc_sb_csum(mdp_super_t * sb)
+static unsigned int calc_sb_csum(mdp_super_t *sb)
{
u64 newcsum = 0;
u32 *sb32 = (u32*)sb;
newcsum += sb32[i];
csum = (newcsum & 0xffffffff) + (newcsum>>32);
-
#ifdef CONFIG_ALPHA
/* This used to use csum_partial, which was wrong for several
* reasons including that different results are returned on
return csum;
}
-
/*
* Handle superblock details.
* We want to be able to handle multiple superblock formats
EXPORT_SYMBOL(md_check_no_bitmap);
/*
- * load_super for 0.90.0
+ * load_super for 0.90.0
*/
static int super_90_load(struct md_rdev *rdev, struct md_rdev *refdev, int minor_version)
{
ev2 = md_event(refsb);
if (ev1 > ev2)
ret = 1;
- else
+ else
ret = 0;
}
rdev->sectors = rdev->sb_start;
if (sb->state & (1<<MD_SB_CLEAN))
mddev->recovery_cp = MaxSector;
else {
- if (sb->events_hi == sb->cp_events_hi &&
+ if (sb->events_hi == sb->cp_events_hi &&
sb->events_lo == sb->cp_events_lo) {
mddev->recovery_cp = sb->recovery_cp;
} else
++ev1;
if (sb->disks[rdev->desc_nr].state & (
(1<<MD_DISK_SYNC) | (1 << MD_DISK_ACTIVE)))
- if (ev1 < mddev->events)
+ if (ev1 < mddev->events)
return -EINVAL;
} else if (mddev->bitmap) {
/* if adding to array with a bitmap, then we can accept an
struct md_rdev *rdev2;
int next_spare = mddev->raid_disks;
-
/* make rdev->sb match mddev data..
*
* 1/ zero out disks
* version 1 superblock
*/
-static __le32 calc_sb_1_csum(struct mdp_superblock_1 * sb)
+static __le32 calc_sb_1_csum(struct mdp_superblock_1 *sb)
{
__le32 disk_csum;
u32 csum;
ret = read_disk_sb(rdev, 4096);
if (ret) return ret;
-
sb = page_address(rdev->sb_page);
if (sb->magic != cpu_to_le32(MD_SB_MAGIC) ||
for (i=0; i<max_dev;i++)
sb->dev_roles[i] = cpu_to_le16(0xfffe);
-
+
rdev_for_each(rdev2, mddev) {
i = rdev2->desc_nr;
if (test_bit(Faulty, &rdev2->flags))
}
EXPORT_SYMBOL(md_integrity_add_rdev);
-static int bind_rdev_to_array(struct md_rdev * rdev, struct mddev * mddev)
+static int bind_rdev_to_array(struct md_rdev *rdev, struct mddev *mddev)
{
char b[BDEVNAME_SIZE];
struct kobject *ko;
char *s;
int err;
- if (rdev->mddev) {
- MD_BUG();
- return -EINVAL;
- }
-
/* prevent duplicates */
if (find_rdev(mddev, rdev->bdev->bd_dev))
return -EEXIST;
* If it is -1, assign a free number, else
* check number is not in use
*/
+ rcu_read_lock();
if (rdev->desc_nr < 0) {
int choice = 0;
- if (mddev->pers) choice = mddev->raid_disks;
- while (find_rdev_nr(mddev, choice))
+ if (mddev->pers)
+ choice = mddev->raid_disks;
+ while (find_rdev_nr_rcu(mddev, choice))
choice++;
rdev->desc_nr = choice;
} else {
- if (find_rdev_nr(mddev, rdev->desc_nr))
+ if (find_rdev_nr_rcu(mddev, rdev->desc_nr)) {
+ rcu_read_unlock();
return -EBUSY;
+ }
}
+ rcu_read_unlock();
if (mddev->max_disks && rdev->desc_nr >= mddev->max_disks) {
printk(KERN_WARNING "md: %s: array is limited to %d devices\n",
mdname(mddev), mddev->max_disks);
kobject_put(&rdev->kobj);
}
-static void unbind_rdev_from_array(struct md_rdev * rdev)
+static void unbind_rdev_from_array(struct md_rdev *rdev)
{
char b[BDEVNAME_SIZE];
- if (!rdev->mddev) {
- MD_BUG();
- return;
- }
+
bd_unlink_disk_holder(rdev->bdev, rdev->mddev->gendisk);
list_del_rcu(&rdev->same_set);
printk(KERN_INFO "md: unbind<%s>\n", bdevname(rdev->bdev,b));
{
struct block_device *bdev = rdev->bdev;
rdev->bdev = NULL;
- if (!bdev)
- MD_BUG();
blkdev_put(bdev, FMODE_READ|FMODE_WRITE|FMODE_EXCL);
}
void md_autodetect_dev(dev_t dev);
-static void export_rdev(struct md_rdev * rdev)
+static void export_rdev(struct md_rdev *rdev)
{
char b[BDEVNAME_SIZE];
+
printk(KERN_INFO "md: export_rdev(%s)\n",
bdevname(rdev->bdev,b));
- if (rdev->mddev)
- MD_BUG();
md_rdev_clear(rdev);
#ifndef MODULE
if (test_bit(AutoDetected, &rdev->flags))
kobject_put(&rdev->kobj);
}
-static void kick_rdev_from_array(struct md_rdev * rdev)
+static void kick_rdev_from_array(struct md_rdev *rdev)
{
unbind_rdev_from_array(rdev);
export_rdev(rdev);
static void export_array(struct mddev *mddev)
{
- struct md_rdev *rdev, *tmp;
+ struct md_rdev *rdev;
- rdev_for_each_safe(rdev, tmp, mddev) {
- if (!rdev->mddev) {
- MD_BUG();
- continue;
- }
+ while (!list_empty(&mddev->disks)) {
+ rdev = list_first_entry(&mddev->disks, struct md_rdev,
+ same_set);
kick_rdev_from_array(rdev);
}
- if (!list_empty(&mddev->disks))
- MD_BUG();
mddev->raid_disks = 0;
mddev->major_version = 0;
}
-static void print_desc(mdp_disk_t *desc)
-{
- printk(" DISK<N:%d,(%d,%d),R:%d,S:%d>\n", desc->number,
- desc->major,desc->minor,desc->raid_disk,desc->state);
-}
-
-static void print_sb_90(mdp_super_t *sb)
-{
- int i;
-
- printk(KERN_INFO
- "md: SB: (V:%d.%d.%d) ID:<%08x.%08x.%08x.%08x> CT:%08x\n",
- sb->major_version, sb->minor_version, sb->patch_version,
- sb->set_uuid0, sb->set_uuid1, sb->set_uuid2, sb->set_uuid3,
- sb->ctime);
- printk(KERN_INFO "md: L%d S%08d ND:%d RD:%d md%d LO:%d CS:%d\n",
- sb->level, sb->size, sb->nr_disks, sb->raid_disks,
- sb->md_minor, sb->layout, sb->chunk_size);
- printk(KERN_INFO "md: UT:%08x ST:%d AD:%d WD:%d"
- " FD:%d SD:%d CSUM:%08x E:%08lx\n",
- sb->utime, sb->state, sb->active_disks, sb->working_disks,
- sb->failed_disks, sb->spare_disks,
- sb->sb_csum, (unsigned long)sb->events_lo);
-
- printk(KERN_INFO);
- for (i = 0; i < MD_SB_DISKS; i++) {
- mdp_disk_t *desc;
-
- desc = sb->disks + i;
- if (desc->number || desc->major || desc->minor ||
- desc->raid_disk || (desc->state && (desc->state != 4))) {
- printk(" D %2d: ", i);
- print_desc(desc);
- }
- }
- printk(KERN_INFO "md: THIS: ");
- print_desc(&sb->this_disk);
-}
-
-static void print_sb_1(struct mdp_superblock_1 *sb)
-{
- __u8 *uuid;
-
- uuid = sb->set_uuid;
- printk(KERN_INFO
- "md: SB: (V:%u) (F:0x%08x) Array-ID:<%pU>\n"
- "md: Name: \"%s\" CT:%llu\n",
- le32_to_cpu(sb->major_version),
- le32_to_cpu(sb->feature_map),
- uuid,
- sb->set_name,
- (unsigned long long)le64_to_cpu(sb->ctime)
- & MD_SUPERBLOCK_1_TIME_SEC_MASK);
-
- uuid = sb->device_uuid;
- printk(KERN_INFO
- "md: L%u SZ%llu RD:%u LO:%u CS:%u DO:%llu DS:%llu SO:%llu"
- " RO:%llu\n"
- "md: Dev:%08x UUID: %pU\n"
- "md: (F:0x%08x) UT:%llu Events:%llu ResyncOffset:%llu CSUM:0x%08x\n"
- "md: (MaxDev:%u) \n",
- le32_to_cpu(sb->level),
- (unsigned long long)le64_to_cpu(sb->size),
- le32_to_cpu(sb->raid_disks),
- le32_to_cpu(sb->layout),
- le32_to_cpu(sb->chunksize),
- (unsigned long long)le64_to_cpu(sb->data_offset),
- (unsigned long long)le64_to_cpu(sb->data_size),
- (unsigned long long)le64_to_cpu(sb->super_offset),
- (unsigned long long)le64_to_cpu(sb->recovery_offset),
- le32_to_cpu(sb->dev_number),
- uuid,
- sb->devflags,
- (unsigned long long)le64_to_cpu(sb->utime) & MD_SUPERBLOCK_1_TIME_SEC_MASK,
- (unsigned long long)le64_to_cpu(sb->events),
- (unsigned long long)le64_to_cpu(sb->resync_offset),
- le32_to_cpu(sb->sb_csum),
- le32_to_cpu(sb->max_dev)
- );
-}
-
-static void print_rdev(struct md_rdev *rdev, int major_version)
-{
- char b[BDEVNAME_SIZE];
- printk(KERN_INFO "md: rdev %s, Sect:%08llu F:%d S:%d DN:%u\n",
- bdevname(rdev->bdev, b), (unsigned long long)rdev->sectors,
- test_bit(Faulty, &rdev->flags), test_bit(In_sync, &rdev->flags),
- rdev->desc_nr);
- if (rdev->sb_loaded) {
- printk(KERN_INFO "md: rdev superblock (MJ:%d):\n", major_version);
- switch (major_version) {
- case 0:
- print_sb_90(page_address(rdev->sb_page));
- break;
- case 1:
- print_sb_1(page_address(rdev->sb_page));
- break;
- }
- } else
- printk(KERN_INFO "md: no rdev superblock!\n");
-}
-
-static void md_print_devices(void)
-{
- struct list_head *tmp;
- struct md_rdev *rdev;
- struct mddev *mddev;
- char b[BDEVNAME_SIZE];
-
- printk("\n");
- printk("md: **********************************\n");
- printk("md: * <COMPLETE RAID STATE PRINTOUT> *\n");
- printk("md: **********************************\n");
- for_each_mddev(mddev, tmp) {
-
- if (mddev->bitmap)
- bitmap_print_sb(mddev->bitmap);
- else
- printk("%s: ", mdname(mddev));
- rdev_for_each(rdev, mddev)
- printk("<%s>", bdevname(rdev->bdev,b));
- printk("\n");
-
- rdev_for_each(rdev, mddev)
- print_rdev(rdev, mddev->major_version);
- }
- printk("md: **********************************\n");
- printk("\n");
-}
-
-
-static void sync_sbs(struct mddev * mddev, int nospares)
+static void sync_sbs(struct mddev *mddev, int nospares)
{
/* Update each superblock (in-memory image), but
* if we are allowed to, skip spares which already
}
}
-static void md_update_sb(struct mddev * mddev, int force_change)
+static void md_update_sb(struct mddev *mddev, int force_change)
{
struct md_rdev *rdev;
int sync_req;
mddev->curr_resync_completed > rdev->recovery_offset)
rdev->recovery_offset = mddev->curr_resync_completed;
- }
+ }
if (!mddev->persistent) {
clear_bit(MD_CHANGE_CLEAN, &mddev->flags);
clear_bit(MD_CHANGE_DEVS, &mddev->flags);
mddev->can_decrease_events = nospares;
}
- if (!mddev->events) {
- /*
- * oops, this 64-bit counter should never wrap.
- * Either we are in around ~1 trillion A.C., assuming
- * 1 reboot per second, or we have a bug:
- */
- MD_BUG();
- mddev->events --;
- }
+ /*
+ * This 64-bit counter should never wrap.
+ * Either we are in around ~1 trillion A.C., assuming
+ * 1 reboot per second, or we have a bug...
+ */
+ WARN_ON(mddev->events == 0);
rdev_for_each(rdev, mddev) {
if (rdev->badblocks.changed)
set_bit(In_sync, &rdev->flags);
err = 0;
} else if (cmd_match(buf, "-insync") && rdev->raid_disk >= 0) {
- clear_bit(In_sync, &rdev->flags);
- rdev->saved_raid_disk = rdev->raid_disk;
- rdev->raid_disk = -1;
- err = 0;
+ if (rdev->mddev->pers == NULL) {
+ clear_bit(In_sync, &rdev->flags);
+ rdev->saved_raid_disk = rdev->raid_disk;
+ rdev->raid_disk = -1;
+ err = 0;
+ }
} else if (cmd_match(buf, "write_error")) {
set_bit(WriteErrorSeen, &rdev->flags);
err = 0;
return len;
}
-
static struct rdev_sysfs_entry rdev_slot =
__ATTR(slot, S_IRUGO|S_IWUSR, slot_show, slot_store);
rdev->sectors = sectors;
if (sectors > oldsectors && my_mddev->external) {
- /* need to check that all other rdevs with the same ->bdev
- * do not overlap. We need to unlock the mddev to avoid
- * a deadlock. We have already changed rdev->sectors, and if
- * we have to change it back, we will have the lock again.
+ /* Need to check that all other rdevs with the same
+ * ->bdev do not overlap. 'rcu' is sufficient to walk
+ * the rdev lists safely.
+ * This check does not provide a hard guarantee, it
+ * just helps avoid dangerous mistakes.
*/
struct mddev *mddev;
int overlap = 0;
struct list_head *tmp;
- mddev_unlock(my_mddev);
+ rcu_read_lock();
for_each_mddev(mddev, tmp) {
struct md_rdev *rdev2;
- mddev_lock_nointr(mddev);
rdev_for_each(rdev2, mddev)
if (rdev->bdev == rdev2->bdev &&
rdev != rdev2 &&
overlap = 1;
break;
}
- mddev_unlock(mddev);
if (overlap) {
mddev_put(mddev);
break;
}
}
- mddev_lock_nointr(my_mddev);
+ rcu_read_unlock();
if (overlap) {
/* Someone else could have slipped in a size
* change here, but doing so is just silly.
static struct rdev_sysfs_entry rdev_size =
__ATTR(size, S_IRUGO|S_IWUSR, rdev_size_show, rdev_size_store);
-
static ssize_t recovery_start_show(struct md_rdev *rdev, char *page)
{
unsigned long long recovery_start = rdev->recovery_offset;
static struct rdev_sysfs_entry rdev_recovery_start =
__ATTR(recovery_start, S_IRUGO|S_IWUSR, recovery_start_show, recovery_start_store);
-
static ssize_t
badblocks_show(struct badblocks *bb, char *page, int unack);
static ssize_t
static struct rdev_sysfs_entry rdev_bad_blocks =
__ATTR(bad_blocks, S_IRUGO|S_IWUSR, bb_show, bb_store);
-
static ssize_t ubb_show(struct md_rdev *rdev, char *page)
{
return badblocks_show(&rdev->badblocks, page, 1);
size = i_size_read(rdev->bdev->bd_inode) >> BLOCK_SIZE_BITS;
if (!size) {
- printk(KERN_WARNING
+ printk(KERN_WARNING
"md: %s has zero or unknown size, marking faulty!\n",
bdevname(rdev->bdev,b));
err = -EINVAL;
goto abort_free;
}
if (err < 0) {
- printk(KERN_WARNING
+ printk(KERN_WARNING
"md: could not read %s's sb, not importing!\n",
bdevname(rdev->bdev,b));
goto abort_free;
* Check a full RAID array for plausibility
*/
-
-static void analyze_sbs(struct mddev * mddev)
+static void analyze_sbs(struct mddev *mddev)
{
int i;
struct md_rdev *rdev, *freshest, *tmp;
default:
printk( KERN_ERR \
"md: fatal superblock inconsistency in %s"
- " -- removing from array\n",
+ " -- removing from array\n",
bdevname(rdev->bdev,b));
kick_rdev_from_array(rdev);
}
-
super_types[mddev->major_version].
validate_super(mddev, freshest);
/* Read a fixed-point number.
* Numbers in sysfs attributes should be in "standard" units where
* possible, so time should be in seconds.
- * However we internally use a a much smaller unit such as
+ * However we internally use a a much smaller unit such as
* milliseconds or jiffies.
* This function takes a decimal number with a possible fractional
* component, and produces an integer which is the result of
return 0;
}
-
static void md_safemode_timeout(unsigned long data);
static ssize_t
/* Looks like we have a winner */
mddev_suspend(mddev);
mddev->pers->stop(mddev);
-
+
if (mddev->pers->sync_request == NULL &&
pers->sync_request != NULL) {
/* need to add the md_redundancy_group */
"md: cannot register extra attributes for %s\n",
mdname(mddev));
mddev->sysfs_action = sysfs_get_dirent(mddev->kobj.sd, "sync_action");
- }
+ }
if (mddev->pers->sync_request != NULL &&
pers->sync_request == NULL) {
/* need to remove the md_redundancy_group */
static struct md_sysfs_entry md_level =
__ATTR(level, S_IRUGO|S_IWUSR, level_show, level_store);
-
static ssize_t
layout_show(struct mddev *mddev, char *page)
{
static struct md_sysfs_entry md_layout =
__ATTR(layout, S_IRUGO|S_IWUSR, layout_show, layout_store);
-
static ssize_t
raid_disks_show(struct mddev *mddev, char *page)
{
return sprintf(page, "%s\n", array_states[st]);
}
-static int do_md_stop(struct mddev * mddev, int ro, struct block_device *bdev);
-static int md_set_readonly(struct mddev * mddev, struct block_device *bdev);
-static int do_md_run(struct mddev * mddev);
+static int do_md_stop(struct mddev *mddev, int ro, struct block_device *bdev);
+static int md_set_readonly(struct mddev *mddev, struct block_device *bdev);
+static int do_md_run(struct mddev *mddev);
static int restart_array(struct mddev *mddev);
static ssize_t
minor != MINOR(dev))
return -EOVERFLOW;
-
if (mddev->persistent) {
rdev = md_import_device(dev, mddev->major_version,
mddev->minor_version);
static struct md_sysfs_entry md_size =
__ATTR(component_size, S_IRUGO|S_IWUSR, size_show, size_store);
-
/* Metadata version.
* This is one of
* 'none' for arrays with no metadata (good luck...)
unsigned long long new = simple_strtoull(buf, &e, 10);
unsigned long long old = mddev->suspend_lo;
- if (mddev->pers == NULL ||
+ if (mddev->pers == NULL ||
mddev->pers->quiesce == NULL)
return -EINVAL;
if (buf == e || (*e && *e != '\n'))
static struct md_sysfs_entry md_suspend_lo =
__ATTR(suspend_lo, S_IRUGO|S_IWUSR, suspend_lo_show, suspend_lo_store);
-
static ssize_t
suspend_hi_show(struct mddev *mddev, char *page)
{
.attrs = md_redundancy_attrs,
};
-
static ssize_t
md_attr_show(struct kobject *kobj, struct attribute *attr, char *page)
{
} else if (mddev->ro == 2) /* auto-readonly not meaningful */
mddev->ro = 0;
- atomic_set(&mddev->writes_pending,0);
+ atomic_set(&mddev->writes_pending,0);
atomic_set(&mddev->max_corr_read_errors,
MD_DEFAULT_MAX_CORRECTED_READ_ERRORS);
mddev->safemode = 0;
if (rdev->raid_disk >= 0)
if (sysfs_link_rdev(mddev, rdev))
/* failure here is OK */;
-
+
set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
-
+
if (mddev->flags & MD_UPDATE_SB_FLAGS)
md_update_sb(mddev, 0);
mddev_lock_nointr(mddev);
mutex_lock(&mddev->open_mutex);
- if (atomic_read(&mddev->openers) > !!bdev ||
+ if ((mddev->pers && atomic_read(&mddev->openers) > !!bdev) ||
mddev->sync_thread ||
(bdev && !test_bit(MD_STILL_CLOSED, &mddev->flags))) {
printk("md: %s still in use.\n",mdname(mddev));
* 0 - completely stop and dis-assemble array
* 2 - stop but do not disassemble array
*/
-static int do_md_stop(struct mddev * mddev, int mode,
+static int do_md_stop(struct mddev *mddev, int mode,
struct block_device *bdev)
{
struct gendisk *disk = mddev->gendisk;
mddev_lock_nointr(mddev);
mutex_lock(&mddev->open_mutex);
- if (atomic_read(&mddev->openers) > !!bdev ||
+ if ((mddev->pers && atomic_read(&mddev->openers) > !!bdev) ||
mddev->sysfs_active ||
mddev->sync_thread ||
(bdev && !test_bit(MD_STILL_CLOSED, &mddev->flags))) {
"md: cannot allocate memory for md drive.\n");
break;
}
- if (mddev_lock(mddev))
+ if (mddev_lock(mddev))
printk(KERN_WARNING "md: %s locked, cannot run\n",
mdname(mddev));
else if (mddev->raid_disks || mddev->major_version
|| !list_empty(&mddev->disks)) {
- printk(KERN_WARNING
+ printk(KERN_WARNING
"md: %s already running, cannot run %s\n",
mdname(mddev), bdevname(rdev0->bdev,b));
mddev_unlock(mddev);
}
#endif /* !MODULE */
-static int get_version(void __user * arg)
+static int get_version(void __user *arg)
{
mdu_version_t ver;
return 0;
}
-static int get_array_info(struct mddev * mddev, void __user * arg)
+static int get_array_info(struct mddev *mddev, void __user *arg)
{
mdu_array_info_t info;
int nr,working,insync,failed,spare;
else {
working++;
if (test_bit(In_sync, &rdev->flags))
- insync++;
+ insync++;
else
spare++;
}
return 0;
}
-static int get_bitmap_file(struct mddev * mddev, void __user * arg)
+static int get_bitmap_file(struct mddev *mddev, void __user * arg)
{
mdu_bitmap_file_t *file = NULL; /* too big for stack allocation */
char *ptr, *buf = NULL;
return err;
}
-static int get_disk_info(struct mddev * mddev, void __user * arg)
+static int get_disk_info(struct mddev *mddev, void __user * arg)
{
mdu_disk_info_t info;
struct md_rdev *rdev;
return 0;
}
-static int add_new_disk(struct mddev * mddev, mdu_disk_info_t *info)
+static int add_new_disk(struct mddev *mddev, mdu_disk_info_t *info)
{
char b[BDEVNAME_SIZE], b2[BDEVNAME_SIZE];
struct md_rdev *rdev;
/* expecting a device which has a superblock */
rdev = md_import_device(dev, mddev->major_version, mddev->minor_version);
if (IS_ERR(rdev)) {
- printk(KERN_WARNING
+ printk(KERN_WARNING
"md: md_import_device returned %ld\n",
PTR_ERR(rdev));
return PTR_ERR(rdev);
err = super_types[mddev->major_version]
.load_super(rdev, rdev0, mddev->minor_version);
if (err < 0) {
- printk(KERN_WARNING
+ printk(KERN_WARNING
"md: %s has different UUID to %s\n",
- bdevname(rdev->bdev,b),
+ bdevname(rdev->bdev,b),
bdevname(rdev0->bdev,b2));
export_rdev(rdev);
return -EINVAL;
if (mddev->pers) {
int err;
if (!mddev->pers->hot_add_disk) {
- printk(KERN_WARNING
+ printk(KERN_WARNING
"%s: personality does not support diskops!\n",
mdname(mddev));
return -EINVAL;
else
rdev = md_import_device(dev, -1, -1);
if (IS_ERR(rdev)) {
- printk(KERN_WARNING
+ printk(KERN_WARNING
"md: md_import_device returned %ld\n",
PTR_ERR(rdev));
return PTR_ERR(rdev);
int err;
rdev = md_import_device(dev, -1, 0);
if (IS_ERR(rdev)) {
- printk(KERN_WARNING
+ printk(KERN_WARNING
"md: error, md_import_device() returned %ld\n",
PTR_ERR(rdev));
return PTR_ERR(rdev);
return 0;
}
-static int hot_remove_disk(struct mddev * mddev, dev_t dev)
+static int hot_remove_disk(struct mddev *mddev, dev_t dev)
{
char b[BDEVNAME_SIZE];
struct md_rdev *rdev;
return -EBUSY;
}
-static int hot_add_disk(struct mddev * mddev, dev_t dev)
+static int hot_add_disk(struct mddev *mddev, dev_t dev)
{
char b[BDEVNAME_SIZE];
int err;
return -EINVAL;
}
if (!mddev->pers->hot_add_disk) {
- printk(KERN_WARNING
+ printk(KERN_WARNING
"%s: personality does not support diskops!\n",
mdname(mddev));
return -EINVAL;
rdev = md_import_device(dev, -1, 0);
if (IS_ERR(rdev)) {
- printk(KERN_WARNING
+ printk(KERN_WARNING
"md: error, md_import_device() returned %ld\n",
PTR_ERR(rdev));
return -EINVAL;
rdev->sectors = rdev->sb_start;
if (test_bit(Faulty, &rdev->flags)) {
- printk(KERN_WARNING
+ printk(KERN_WARNING
"md: can not hot-add faulty %s disk to %s!\n",
bdevname(rdev->bdev,b), mdname(mddev));
err = -EINVAL;
/* we should be able to change the bitmap.. */
}
-
if (fd >= 0) {
struct inode *inode;
if (mddev->bitmap)
* The minor and patch _version numbers are also kept incase the
* super_block handler wishes to interpret them.
*/
-static int set_array_info(struct mddev * mddev, mdu_array_info_t *info)
+static int set_array_info(struct mddev *mddev, mdu_array_info_t *info)
{
if (info->raid_disks == 0) {
info->major_version >= ARRAY_SIZE(super_types) ||
super_types[info->major_version].name == NULL) {
/* maybe try to auto-load a module? */
- printk(KERN_INFO
+ printk(KERN_INFO
"md: superblock version %d not known\n",
info->major_version);
return -EINVAL;
return rv;
}
-
/*
* update_array_info is used to change the configuration of an
* on-line array.
case GET_DISK_INFO:
case HOT_ADD_DISK:
case HOT_REMOVE_DISK:
- case PRINT_RAID_DEBUG:
case RAID_AUTORUN:
case RAID_VERSION:
case RESTART_ARRAY_RW:
switch (cmd) {
case RAID_VERSION:
err = get_version(argp);
- goto done;
-
- case PRINT_RAID_DEBUG:
- err = 0;
- md_print_devices();
- goto done;
+ goto out;
#ifndef MODULE
case RAID_AUTORUN:
err = 0;
autostart_arrays(arg);
- goto done;
+ goto out;
#endif
default:;
}
if (!mddev) {
BUG();
- goto abort;
+ goto out;
}
/* Some actions do not requires the mutex */
err = -ENODEV;
else
err = get_array_info(mddev, argp);
- goto abort;
+ goto out;
case GET_DISK_INFO:
if (!mddev->raid_disks && !mddev->external)
err = -ENODEV;
else
err = get_disk_info(mddev, argp);
- goto abort;
+ goto out;
case SET_DISK_FAULTY:
err = set_disk_faulty(mddev, new_decode_dev(arg));
- goto abort;
+ goto out;
}
if (cmd == ADD_NEW_DISK)
* and writes
*/
mutex_lock(&mddev->open_mutex);
- if (atomic_read(&mddev->openers) > 1) {
+ if (mddev->pers && atomic_read(&mddev->openers) > 1) {
mutex_unlock(&mddev->open_mutex);
err = -EBUSY;
- goto abort;
+ goto out;
}
set_bit(MD_STILL_CLOSED, &mddev->flags);
mutex_unlock(&mddev->open_mutex);
}
err = mddev_lock(mddev);
if (err) {
- printk(KERN_INFO
+ printk(KERN_INFO
"md: ioctl lock interrupted, reason %d, cmd %d\n",
err, cmd);
- goto abort;
+ goto out;
}
if (cmd == SET_ARRAY_INFO) {
memset(&info, 0, sizeof(info));
else if (copy_from_user(&info, argp, sizeof(info))) {
err = -EFAULT;
- goto abort_unlock;
+ goto unlock;
}
if (mddev->pers) {
err = update_array_info(mddev, &info);
if (err) {
printk(KERN_WARNING "md: couldn't update"
" array info. %d\n", err);
- goto abort_unlock;
+ goto unlock;
}
- goto done_unlock;
+ goto unlock;
}
if (!list_empty(&mddev->disks)) {
printk(KERN_WARNING
"md: array %s already has disks!\n",
mdname(mddev));
err = -EBUSY;
- goto abort_unlock;
+ goto unlock;
}
if (mddev->raid_disks) {
printk(KERN_WARNING
"md: array %s already initialised!\n",
mdname(mddev));
err = -EBUSY;
- goto abort_unlock;
+ goto unlock;
}
err = set_array_info(mddev, &info);
if (err) {
printk(KERN_WARNING "md: couldn't set"
" array info. %d\n", err);
- goto abort_unlock;
+ goto unlock;
}
- goto done_unlock;
+ goto unlock;
}
/*
&& cmd != RUN_ARRAY && cmd != SET_BITMAP_FILE
&& cmd != GET_BITMAP_FILE) {
err = -ENODEV;
- goto abort_unlock;
+ goto unlock;
}
/*
switch (cmd) {
case GET_BITMAP_FILE:
err = get_bitmap_file(mddev, argp);
- goto done_unlock;
+ goto unlock;
case RESTART_ARRAY_RW:
err = restart_array(mddev);
- goto done_unlock;
+ goto unlock;
case STOP_ARRAY:
err = do_md_stop(mddev, 0, bdev);
- goto done_unlock;
+ goto unlock;
case STOP_ARRAY_RO:
err = md_set_readonly(mddev, bdev);
- goto done_unlock;
+ goto unlock;
case HOT_REMOVE_DISK:
err = hot_remove_disk(mddev, new_decode_dev(arg));
- goto done_unlock;
+ goto unlock;
case ADD_NEW_DISK:
/* We can support ADD_NEW_DISK on read-only arrays
break;
else
err = add_new_disk(mddev, &info);
- goto done_unlock;
+ goto unlock;
}
break;
case BLKROSET:
if (get_user(ro, (int __user *)(arg))) {
err = -EFAULT;
- goto done_unlock;
+ goto unlock;
}
err = -EINVAL;
* does not matter, no writes are coming
*/
if (ro)
- goto done_unlock;
+ goto unlock;
/* are we are already prepared for writes? */
if (mddev->ro != 1)
- goto done_unlock;
+ goto unlock;
/* transitioning to readauto need only happen for
* arrays that call md_write_start
set_disk_ro(mddev->gendisk, 0);
}
}
- goto done_unlock;
+ goto unlock;
}
/*
* The remaining ioctls are changing the state of the
* superblock, so we do not allow them on read-only arrays.
- * However non-MD ioctls (e.g. get-size) will still come through
- * here and hit the 'default' below, so only disallow
- * 'md' ioctls, and switch to rw mode if started auto-readonly.
*/
- if (_IOC_TYPE(cmd) == MD_MAJOR && mddev->ro && mddev->pers) {
+ if (mddev->ro && mddev->pers) {
if (mddev->ro == 2) {
mddev->ro = 0;
sysfs_notify_dirent_safe(mddev->sysfs_state);
}
} else {
err = -EROFS;
- goto abort_unlock;
+ goto unlock;
}
}
err = -EFAULT;
else
err = add_new_disk(mddev, &info);
- goto done_unlock;
+ goto unlock;
}
case HOT_ADD_DISK:
err = hot_add_disk(mddev, new_decode_dev(arg));
- goto done_unlock;
+ goto unlock;
case RUN_ARRAY:
err = do_md_run(mddev);
- goto done_unlock;
+ goto unlock;
case SET_BITMAP_FILE:
err = set_bitmap_file(mddev, (int)arg);
- goto done_unlock;
+ goto unlock;
default:
err = -EINVAL;
- goto abort_unlock;
+ goto unlock;
}
-done_unlock:
-abort_unlock:
+unlock:
if (mddev->hold_active == UNTIL_IOCTL &&
err != -EINVAL)
mddev->hold_active = 0;
mddev_unlock(mddev);
-
- return err;
-done:
- if (err)
- MD_BUG();
-abort:
+out:
return err;
}
#ifdef CONFIG_COMPAT
static void md_release(struct gendisk *disk, fmode_t mode)
{
- struct mddev *mddev = disk->private_data;
+ struct mddev *mddev = disk->private_data;
BUG_ON(!mddev);
atomic_dec(&mddev->openers);
.revalidate_disk= md_revalidate,
};
-static int md_thread(void * arg)
+static int md_thread(void *arg)
{
struct md_thread *thread = arg;
wake_up(&thread->wqueue);
}
}
+EXPORT_SYMBOL(md_wakeup_thread);
struct md_thread *md_register_thread(void (*run) (struct md_thread *),
struct mddev *mddev, const char *name)
}
return thread;
}
+EXPORT_SYMBOL(md_register_thread);
void md_unregister_thread(struct md_thread **threadp)
{
kthread_stop(thread->tsk);
kfree(thread);
}
+EXPORT_SYMBOL(md_unregister_thread);
void md_error(struct mddev *mddev, struct md_rdev *rdev)
{
- if (!mddev) {
- MD_BUG();
- return;
- }
-
if (!rdev || test_bit(Faulty, &rdev->flags))
return;
queue_work(md_misc_wq, &mddev->event_work);
md_new_event_inintr(mddev);
}
+EXPORT_SYMBOL(md_error);
/* seq_file implementation /proc/mdstat */
seq_printf(seq, "\n");
}
-
-static void status_resync(struct seq_file *seq, struct mddev * mddev)
+static void status_resync(struct seq_file *seq, struct mddev *mddev)
{
sector_t max_sectors, resync, res;
unsigned long dt, db;
else
max_sectors = mddev->dev_sectors;
- /*
- * Should not happen.
- */
- if (!max_sectors) {
- MD_BUG();
- return;
- }
+ WARN_ON(max_sectors == 0);
/* Pick 'scale' such that (resync>>scale)*1000 will fit
* in a sector_t, and (max_sectors>>scale) will fit in a
* u32, as those are the requirements for sector_div.
{
struct list_head *tmp;
struct mddev *next_mddev, *mddev = v;
-
+
++*pos;
if (v == (void*)2)
return NULL;
else {
next_mddev = (void*)2;
*pos = 0x10000;
- }
+ }
spin_unlock(&all_mddevs_lock);
if (v != (void*)1)
if (mddev->pers) {
mddev->pers->status(seq, mddev);
- seq_printf(seq, "\n ");
+ seq_printf(seq, "\n ");
if (mddev->pers->sync_request) {
if (mddev->curr_resync > 2) {
status_resync(seq, mddev);
seq_printf(seq, "\n");
}
mddev_unlock(mddev);
-
+
return 0;
}
int register_md_personality(struct md_personality *p)
{
+ printk(KERN_INFO "md: %s personality registered for level %d\n",
+ p->name, p->level);
spin_lock(&pers_lock);
list_add_tail(&p->list, &pers_list);
- printk(KERN_INFO "md: %s personality registered for level %d\n", p->name, p->level);
spin_unlock(&pers_lock);
return 0;
}
+EXPORT_SYMBOL(register_md_personality);
int unregister_md_personality(struct md_personality *p)
{
spin_unlock(&pers_lock);
return 0;
}
+EXPORT_SYMBOL(unregister_md_personality);
static int is_mddev_idle(struct mddev *mddev, int init)
{
- struct md_rdev * rdev;
+ struct md_rdev *rdev;
int idle;
int curr_events;
// stop recovery, signal do_sync ....
}
}
-
+EXPORT_SYMBOL(md_done_sync);
/* md_write_start(mddev, bi)
* If we need to update some array metadata (e.g. 'active' flag
wait_event(mddev->sb_wait,
!test_bit(MD_CHANGE_PENDING, &mddev->flags));
}
+EXPORT_SYMBOL(md_write_start);
void md_write_end(struct mddev *mddev)
{
mod_timer(&mddev->safemode_timer, jiffies + mddev->safemode_delay);
}
}
+EXPORT_SYMBOL(md_write_end);
/* md_allow_write(mddev)
* Calling this ensures that the array is marked 'active' so that writes
return spares;
}
+static void md_start_sync(struct work_struct *ws)
+{
+ struct mddev *mddev = container_of(ws, struct mddev, del_work);
+
+ mddev->sync_thread = md_register_thread(md_do_sync,
+ mddev,
+ "resync");
+ if (!mddev->sync_thread) {
+ printk(KERN_ERR "%s: could not start resync"
+ " thread...\n",
+ mdname(mddev));
+ /* leave the spares where they are, it shouldn't hurt */
+ clear_bit(MD_RECOVERY_SYNC, &mddev->recovery);
+ clear_bit(MD_RECOVERY_RESHAPE, &mddev->recovery);
+ clear_bit(MD_RECOVERY_REQUESTED, &mddev->recovery);
+ clear_bit(MD_RECOVERY_CHECK, &mddev->recovery);
+ clear_bit(MD_RECOVERY_RUNNING, &mddev->recovery);
+ if (test_and_clear_bit(MD_RECOVERY_RECOVER,
+ &mddev->recovery))
+ if (mddev->sysfs_action)
+ sysfs_notify_dirent_safe(mddev->sysfs_action);
+ } else
+ md_wakeup_thread(mddev->sync_thread);
+ sysfs_notify_dirent_safe(mddev->sysfs_action);
+ md_new_event(mddev);
+}
+
/*
* This routine is regularly called by all per-raid-array threads to
* deal with generic issues like resync and super-block update.
if (!test_and_clear_bit(MD_RECOVERY_NEEDED, &mddev->recovery) ||
test_bit(MD_RECOVERY_FROZEN, &mddev->recovery))
- goto unlock;
+ goto not_running;
/* no recovery is running.
* remove any failed drives, then
* add spares if possible.
if (mddev->pers->check_reshape == NULL ||
mddev->pers->check_reshape(mddev) != 0)
/* Cannot proceed */
- goto unlock;
+ goto not_running;
set_bit(MD_RECOVERY_RESHAPE, &mddev->recovery);
clear_bit(MD_RECOVERY_RECOVER, &mddev->recovery);
} else if ((spares = remove_and_add_spares(mddev, NULL))) {
clear_bit(MD_RECOVERY_RECOVER, &mddev->recovery);
} else if (!test_bit(MD_RECOVERY_SYNC, &mddev->recovery))
/* nothing to be done ... */
- goto unlock;
+ goto not_running;
if (mddev->pers->sync_request) {
if (spares) {
*/
bitmap_write_all(mddev->bitmap);
}
- mddev->sync_thread = md_register_thread(md_do_sync,
- mddev,
- "resync");
- if (!mddev->sync_thread) {
- printk(KERN_ERR "%s: could not start resync"
- " thread...\n",
- mdname(mddev));
- /* leave the spares where they are, it shouldn't hurt */
- clear_bit(MD_RECOVERY_RUNNING, &mddev->recovery);
- clear_bit(MD_RECOVERY_SYNC, &mddev->recovery);
- clear_bit(MD_RECOVERY_RESHAPE, &mddev->recovery);
- clear_bit(MD_RECOVERY_REQUESTED, &mddev->recovery);
- clear_bit(MD_RECOVERY_CHECK, &mddev->recovery);
- } else
- md_wakeup_thread(mddev->sync_thread);
- sysfs_notify_dirent_safe(mddev->sysfs_action);
- md_new_event(mddev);
+ INIT_WORK(&mddev->del_work, md_start_sync);
+ queue_work(md_misc_wq, &mddev->del_work);
+ goto unlock;
}
- unlock:
- wake_up(&mddev->sb_wait);
-
+ not_running:
if (!mddev->sync_thread) {
clear_bit(MD_RECOVERY_RUNNING, &mddev->recovery);
if (test_and_clear_bit(MD_RECOVERY_RECOVER,
if (mddev->sysfs_action)
sysfs_notify_dirent_safe(mddev->sysfs_action);
}
+ unlock:
+ wake_up(&mddev->sb_wait);
mddev_unlock(mddev);
}
}
+EXPORT_SYMBOL(md_check_recovery);
void md_reap_sync_thread(struct mddev *mddev)
{
if (mddev->event_work.func)
queue_work(md_misc_wq, &mddev->event_work);
}
+EXPORT_SYMBOL(md_reap_sync_thread);
void md_wait_for_blocked_rdev(struct md_rdev *rdev, struct mddev *mddev)
{
}
}
-
static void autostart_arrays(int part)
{
struct md_rdev *rdev;
if (IS_ERR(rdev))
continue;
- if (test_bit(Faulty, &rdev->flags)) {
- MD_BUG();
+ if (test_bit(Faulty, &rdev->flags))
continue;
- }
+
set_bit(AutoDetected, &rdev->flags);
list_add(&rdev->same_set, &pending_raid_disks);
i_passed++;
module_param_call(start_ro, set_ro, get_ro, NULL, S_IRUSR|S_IWUSR);
module_param(start_dirty_degraded, int, S_IRUGO|S_IWUSR);
-
module_param_call(new_array, add_named_array, NULL, NULL, S_IWUSR);
-EXPORT_SYMBOL(register_md_personality);
-EXPORT_SYMBOL(unregister_md_personality);
-EXPORT_SYMBOL(md_error);
-EXPORT_SYMBOL(md_done_sync);
-EXPORT_SYMBOL(md_write_start);
-EXPORT_SYMBOL(md_write_end);
-EXPORT_SYMBOL(md_register_thread);
-EXPORT_SYMBOL(md_unregister_thread);
-EXPORT_SYMBOL(md_wakeup_thread);
-EXPORT_SYMBOL(md_check_recovery);
-EXPORT_SYMBOL(md_reap_sync_thread);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("MD RAID framework");
MODULE_ALIAS("md");
/*
md.h : kernel internal structure of the Linux MD driver
Copyright (C) 1996-98 Ingo Molnar, Gadi Oxman
-
+
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2, or (at your option)
any later version.
-
+
You should have received a copy of the GNU General Public License
(for example /usr/src/linux/COPYING); if not, write to the Free
- Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#ifndef _MD_MD_H
__u64 sb_events;
sector_t data_offset; /* start of data in array */
sector_t new_data_offset;/* only relevant while reshaping */
- sector_t sb_start; /* offset of the super block (in 512byte sectors) */
+ sector_t sb_start; /* offset of the super block (in 512byte sectors) */
int sb_size; /* bytes in the superblock */
int preferred_minor; /* autorun support */
minor_version,
patch_version;
int persistent;
- int external; /* metadata is
+ int external; /* metadata is
* managed externally */
char metadata_type[17]; /* externally set*/
int chunk_sectors;
char clevel[16];
int raid_disks;
int max_disks;
- sector_t dev_sectors; /* used size of
+ sector_t dev_sectors; /* used size of
* component devices */
sector_t array_sectors; /* exported array size */
int external_size; /* size managed
int parallel_resync;
int ok_start_degraded;
- /* recovery/resync flags
+ /* recovery/resync flags
* NEEDED: we might need to start a resync/recover
* RUNNING: a thread is running, or about to be started
* SYNC: actually doing a resync, not a recovery
unsigned int safemode; /* if set, update "clean" superblock
* when no writes pending.
- */
+ */
unsigned int safemode_delay;
struct timer_list safemode_timer;
- atomic_t writes_pending;
+ atomic_t writes_pending;
struct request_queue *queue; /* for plugging ... */
- struct bitmap *bitmap; /* the bitmap for the device */
+ struct bitmap *bitmap; /* the bitmap for the device */
struct {
struct file *file; /* the bitmap file */
loff_t offset; /* offset from superblock of
* start of bitmap. May be
* negative, but not '0'
* For external metadata, offset
- * from start of device.
+ * from start of device.
*/
unsigned long space; /* space available at this offset */
loff_t default_offset; /* this is the offset to use when
int external;
} bitmap_info;
- atomic_t max_corr_read_errors; /* max read retries */
+ atomic_t max_corr_read_errors; /* max read retries */
struct list_head all_mddevs;
struct attribute_group *to_remove;
void (*sync_super)(struct mddev *mddev, struct md_rdev *rdev);
};
-
static inline void rdev_dec_pending(struct md_rdev *rdev, struct mddev *mddev)
{
int faulty = test_bit(Faulty, &rdev->flags);
static inline void md_sync_acct(struct block_device *bdev, unsigned long nr_sectors)
{
- atomic_add(nr_sectors, &bdev->bd_contains->bd_disk->sync_io);
+ atomic_add(nr_sectors, &bdev->bd_contains->bd_disk->sync_io);
}
struct md_personality
int (*stop)(struct mddev *mddev);
void (*status)(struct seq_file *seq, struct mddev *mddev);
/* error_handler must set ->faulty and clear ->in_sync
- * if appropriate, and should abort recovery if needed
+ * if appropriate, and should abort recovery if needed
*/
void (*error_handler)(struct mddev *mddev, struct md_rdev *rdev);
int (*hot_add_disk) (struct mddev *mddev, struct md_rdev *rdev);
void *(*takeover) (struct mddev *mddev);
};
-
struct md_sysfs_entry {
struct attribute attr;
ssize_t (*show)(struct mddev *, char *);
void (*run) (struct md_thread *thread);
struct mddev *mddev;
wait_queue_head_t wqueue;
- unsigned long flags;
+ unsigned long flags;
struct task_struct *tsk;
unsigned long timeout;
void *private;
extern void md_super_write(struct mddev *mddev, struct md_rdev *rdev,
sector_t sector, int size, struct page *page);
extern void md_super_wait(struct mddev *mddev);
-extern int sync_page_io(struct md_rdev *rdev, sector_t sector, int size,
+extern int sync_page_io(struct md_rdev *rdev, sector_t sector, int size,
struct page *page, int rw, bool metadata_op);
extern void md_do_sync(struct md_thread *thread);
extern void md_new_event(struct mddev *mddev);
#define NR_RESERVED_BUFS 32
-
static int multipath_map (struct mpconf *conf)
{
int i, disks = conf->raid_disks;
/*
- * Later we do read balancing on the read side
+ * Later we do read balancing on the read side
* now we use the first available disk.
*/
md_wakeup_thread(mddev->thread);
}
-
/*
* multipath_end_bh_io() is called when we have finished servicing a multipathed
* operation and are ready to return a success/failure code to the buffer
*/
char b[BDEVNAME_SIZE];
md_error (mp_bh->mddev, rdev);
- printk(KERN_ERR "multipath: %s: rescheduling sector %llu\n",
- bdevname(rdev->bdev,b),
+ printk(KERN_ERR "multipath: %s: rescheduling sector %llu\n",
+ bdevname(rdev->bdev,b),
(unsigned long long)bio->bi_iter.bi_sector);
multipath_reschedule_retry(mp_bh);
} else
{
struct mpconf *conf = mddev->private;
int i;
-
+
seq_printf (seq, " [%d/%d] [", conf->raid_disks,
conf->raid_disks - mddev->degraded);
for (i = 0; i < conf->raid_disks; i++)
seq_printf (seq, "%s",
- conf->multipaths[i].rdev &&
+ conf->multipaths[i].rdev &&
test_bit(In_sync, &conf->multipaths[i].rdev->flags) ? "U" : "_");
seq_printf (seq, "]");
}
* first check if this is a queued request for a device
* which has just failed.
*/
- printk(KERN_ALERT
+ printk(KERN_ALERT
"multipath: only one IO path left and IO error.\n");
/* leave it active... it's all we have */
return;
}
}
-
static int multipath_add_disk(struct mddev *mddev, struct md_rdev *rdev)
{
struct mpconf *conf = mddev->private;
return err;
}
-
-
/*
* This is a kernel thread which:
*
bio = &mp_bh->bio;
bio->bi_iter.bi_sector = mp_bh->master_bio->bi_iter.bi_sector;
-
+
if ((mp_bh->path = multipath_map (conf))<0) {
printk(KERN_ALERT "multipath: %s: unrecoverable IO read"
" error for block %llu\n",
conf = kzalloc(sizeof(struct mpconf), GFP_KERNEL);
mddev->private = conf;
if (!conf) {
- printk(KERN_ERR
+ printk(KERN_ERR
"multipath: couldn't allocate memory for %s\n",
mdname(mddev));
goto out;
conf->multipaths = kzalloc(sizeof(struct multipath_info)*mddev->raid_disks,
GFP_KERNEL);
if (!conf->multipaths) {
- printk(KERN_ERR
+ printk(KERN_ERR
"multipath: couldn't allocate memory for %s\n",
mdname(mddev));
goto out_free_conf;
conf->pool = mempool_create_kmalloc_pool(NR_RESERVED_BUFS,
sizeof(struct multipath_bh));
if (conf->pool == NULL) {
- printk(KERN_ERR
+ printk(KERN_ERR
"multipath: couldn't allocate memory for %s\n",
mdname(mddev));
goto out_free_conf;
}
}
- printk(KERN_INFO
+ printk(KERN_INFO
"multipath: array %s active with %d out of %d IO paths\n",
mdname(mddev), conf->raid_disks - mddev->degraded,
mddev->raid_disks);
return -EIO;
}
-
static int multipath_stop (struct mddev *mddev)
{
struct mpconf *conf = mddev->private;
/*
raid0.c : Multiple Devices driver for Linux
- Copyright (C) 1994-96 Marc ZYNGIER
+ Copyright (C) 1994-96 Marc ZYNGIER
<zyngier@ufr-info-p7.ibp.fr> or
<maz@gloups.fdn.fr>
- Copyright (C) 1999, 2000 Ingo Molnar, Red Hat
-
+ Copyright (C) 1999, 2000 Ingo Molnar, Red Hat
RAID-0 management functions.
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2, or (at your option)
any later version.
-
+
You should have received a copy of the GNU General Public License
(for example /usr/src/linux/COPYING); if not, write to the Free
- Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#include <linux/blkdev.h>
bio_put(to_put);
}
-
/*
* This routine returns the disk from which the requested read should
* be done. There is a per-array 'next expected sequential IO' sector
* However if there are already pending
* requests (preventing the barrier from
* rising completely), and the
- * pre-process bio queue isn't empty,
+ * per-process bio queue isn't empty,
* then don't wait, as we need to empty
- * that queue to get the nr_pending
- * count down.
+ * that queue to allow conf->start_next_window
+ * to increase.
*/
wait_event_lock_irq(conf->wait_barrier,
!conf->array_frozen &&
(!conf->barrier ||
- ((conf->start_next_window <
- conf->next_resync + RESYNC_SECTORS) &&
- current->bio_list &&
- !bio_list_empty(current->bio_list))),
+ ((conf->start_next_window <
+ conf->next_resync + RESYNC_SECTORS) &&
+ current->bio_list &&
+ !bio_list_empty(current->bio_list))),
conf->resync_lock);
conf->nr_waiting--;
}
spin_unlock_irq(&conf->resync_lock);
}
-
-/* duplicate the data pages for behind I/O
+/* duplicate the data pages for behind I/O
*/
static void alloc_behind_pages(struct bio *bio, struct r1bio *r1_bio)
{
seq_printf(seq, "]");
}
-
static void error(struct mddev *mddev, struct md_rdev *rdev)
{
char b[BDEVNAME_SIZE];
unsigned long flags;
/*
- * Find all failed disks within the RAID1 configuration
+ * Find all failed disks within the RAID1 configuration
* and mark them readable.
* Called under mddev lock, so rcu protection not needed.
*/
return count;
}
-
static int raid1_add_disk(struct mddev *mddev, struct md_rdev *rdev)
{
struct r1conf *conf = mddev->private;
return err;
}
-
static void end_sync_read(struct bio *bio, int error)
{
struct r1bio *r1_bio = bio->bi_private;
return 1;
}
-static int process_checks(struct r1bio *r1_bio)
+static void process_checks(struct r1bio *r1_bio)
{
/* We have read all readable devices. If we haven't
* got the block, then there is no hope left.
bio_copy_data(sbio, pbio);
}
- return 0;
}
static void sync_request_write(struct mddev *mddev, struct r1bio *r1_bio)
return;
if (test_bit(MD_RECOVERY_REQUESTED, &mddev->recovery))
- if (process_checks(r1_bio) < 0)
- return;
+ process_checks(r1_bio);
+
/*
* schedule writes
*/
blk_finish_plug(&plug);
}
-
static int init_resync(struct r1conf *conf)
{
int buffs;
/* remove last page from this bio */
bio->bi_vcnt--;
bio->bi_iter.bi_size -= len;
- bio->bi_flags &= ~(1<< BIO_SEG_VALID);
+ __clear_bit(BIO_SEG_VALID, &bio->bi_flags);
}
goto bio_full;
}
printk(KERN_NOTICE "md/raid1:%s: not clean"
" -- starting background reconstruction\n",
mdname(mddev));
- printk(KERN_INFO
+ printk(KERN_INFO
"md/raid1:%s: active with %d out of %d mirrors\n",
- mdname(mddev), mddev->raid_disks - mddev->degraded,
+ mdname(mddev), mddev->raid_disks - mddev->degraded,
mddev->raid_disks);
/*
*/
int recovery_disabled;
-
/* poolinfo contains information about the content of the
* mempools - it changes when the array grows or shrinks
*/
*/
struct page *tmppage;
-
/* When taking over an array from a different personality, we store
* the new thread here until we fully activate the array.
*/
struct md_rdev *rdev;
struct r10conf *conf = r10_bio->mddev->private;
-
slot = r10_bio->read_slot;
dev = r10_bio->devs[slot].devnum;
rdev = r10_bio->devs[slot].rdev;
md_write_start(mddev, bio);
-
do {
/*
return count;
}
-
static int raid10_add_disk(struct mddev *mddev, struct md_rdev *rdev)
{
struct r10conf *conf = mddev->private;
return err;
}
-
static void end_sync_read(struct bio *bio, int error)
{
struct r10bio *r10_bio = bio->bi_private;
}
}
-
/*
* Used by fix_read_error() to decay the per rdev read_errors.
* We halve the read error count for every hour that has elapsed
blk_finish_plug(&plug);
}
-
static int init_resync(struct r10conf *conf)
{
int buffs;
/* remove last page from this bio */
bio2->bi_vcnt--;
bio2->bi_iter.bi_size -= len;
- bio2->bi_flags &= ~(1<< BIO_SEG_VALID);
+ __clear_bit(BIO_SEG_VALID, &bio2->bi_flags);
}
goto bio_full;
}
blk_queue_merge_bvec(mddev->queue, raid10_mergeable_bvec);
}
-
if (md_integrity_register(mddev))
goto out_free_conf;
mempool_destroy(conf->r10bio_pool);
safe_put_page(conf->tmppage);
kfree(conf->mirrors);
+ kfree(conf->mirrors_old);
+ kfree(conf->mirrors_new);
kfree(conf);
mddev->private = NULL;
return 0;
memcpy(conf->mirrors_new, conf->mirrors,
sizeof(struct raid10_info)*conf->prev.raid_disks);
smp_mb();
- kfree(conf->mirrors_old); /* FIXME and elsewhere */
+ kfree(conf->mirrors_old);
conf->mirrors_old = conf->mirrors;
conf->mirrors = conf->mirrors_new;
conf->mirrors_new = NULL;
read_bio->bi_end_io = end_sync_read;
read_bio->bi_rw = READ;
read_bio->bi_flags &= (~0UL << BIO_RESET_BITS);
- read_bio->bi_flags |= 1 << BIO_UPTODATE;
+ __set_bit(BIO_UPTODATE, &read_bio->bi_flags);
read_bio->bi_vcnt = 0;
read_bio->bi_iter.bi_size = 0;
r10_bio->master_bio = read_bio;
/* Remove last page from this bio */
bio2->bi_vcnt--;
bio2->bi_iter.bi_size -= len;
- bio2->bi_flags &= ~(1<<BIO_SEG_VALID);
+ __clear_bit(BIO_SEG_VALID, &bio2->bi_flags);
}
goto bio_full;
}
conf->fullsync = 0;
}
-
static int handle_reshape_read_error(struct mddev *mddev,
struct r10bio *r10_bio)
{
hlist_add_head(&sh->hash, hp);
}
-
/* find an idle stripe, make sure it is unhashed, and return it. */
static struct stripe_head *get_free_stripe(struct r5conf *conf, int hash)
{
BUG_ON(stripe_operations_active(sh));
pr_debug("init_stripe called, stripe %llu\n",
- (unsigned long long)sh->sector);
-
- remove_hash(sh);
+ (unsigned long long)sector);
retry:
seq = read_seqcount_begin(&conf->gen_lock);
sh->generation = conf->generation - previous;
stripe_set_idx(sector, conf, previous, sh);
sh->state = 0;
-
for (i = sh->disks; i--; ) {
struct r5dev *dev = &sh->dev[i];
}
}
-
static void ops_complete_prexor(void *stripe_head_ref)
{
struct stripe_head *sh = stripe_head_ref;
return new_sector;
}
-
static sector_t compute_blocknr(struct stripe_head *sh, int i, int previous)
{
struct r5conf *conf = sh->raid_conf;
sector_t r_sector;
struct stripe_head sh2;
-
chunk_offset = sector_div(new_sector, sectors_per_chunk);
stripe = new_sector;
return r_sector;
}
-
static void
schedule_reconstruction(struct stripe_head *sh, struct stripe_head_state *s,
int rcw, int expand)
set_bit(STRIPE_HANDLE, &sh->state);
}
-
/* handle_stripe_clean_event
* any written block on an uptodate or failed drive can be returned.
* Note that if we 'wrote' to a failed drive, it will be UPTODATE, but
}
}
-
static void handle_parity_checks6(struct r5conf *conf, struct stripe_head *sh,
struct stripe_head_state *s,
int disks)
}
}
-
/* Finish reconstruct operations initiated by the expansion process */
if (sh->reconstruct_state == reconstruct_state_result) {
struct stripe_head *sh_src
return max;
}
-
static int in_chunk_boundary(struct mddev *mddev, struct bio *bio)
{
sector_t sector = bio->bi_iter.bi_sector + get_start_sect(bio->bi_bdev);
md_wakeup_thread(conf->mddev->thread);
}
-
static struct bio *remove_bio_from_retry(struct r5conf *conf)
{
struct bio *bi;
return bi;
}
-
/*
* The "raid5_align_endio" should check if the read succeeded and if it
* did, call bio_endio on the original bio (having bio_put the new bio
return;
}
-
pr_debug("raid5_align_endio : io error...handing IO for a retry\n");
add_bio_to_retry(raid_bi, conf);
return 1;
}
-
static int chunk_aligned_read(struct mddev *mddev, struct bio * raid_bio)
{
struct r5conf *conf = mddev->private;
rcu_read_unlock();
raid_bio->bi_next = (void*)rdev;
align_bi->bi_bdev = rdev->bdev;
- align_bi->bi_flags &= ~(1 << BIO_SEG_VALID);
+ __clear_bit(BIO_SEG_VALID, &align_bi->bi_flags);
if (!bio_fits_rdev(align_bi) ||
is_badblock(rdev, align_bi->bi_iter.bi_sector,
raid5_show_skip_copy,
raid5_store_skip_copy);
-
static ssize_t
stripe_cache_active_show(struct mddev *mddev, char *page)
{
return ERR_PTR(-ENOMEM);
}
-
static int only_parity(int raid_disk, int algo, int raid_disks, int max_degraded)
{
switch (algo) {
return 1;
break;
case ALGORITHM_PARITY_0_6:
- if (raid_disk == 0 ||
+ if (raid_disk == 0 ||
raid_disk == raid_disks - 1)
return 1;
break;
"reshape");
}
-
/* Ok, everything is just fine now */
if (mddev->to_remove == &raid5_attrs_group)
mddev->to_remove = NULL;
}
}
-
static void *raid45_takeover_raid0(struct mddev *mddev, int level)
{
struct r0conf *raid0_conf = mddev->private;
return setup_conf(mddev);
}
-
static void *raid5_takeover_raid1(struct mddev *mddev)
{
int chunksect;
return setup_conf(mddev);
}
-
static int raid5_check_reshape(struct mddev *mddev)
{
/* For a 2-drive array, the layout and chunk size can be changed
return setup_conf(mddev);
}
-
static struct md_personality raid6_personality =
{
.name = "raid6",
*/
/*
- * Operations state - intermediate states that are visible outside of
+ * Operations state - intermediate states that are visible outside of
* STRIPE_ACTIVE.
* In general _idle indicates nothing is running, _run indicates a data
* processing operation is active, and _result means the data processing result
* HANDLE gets cleared if stripe_handle leaves nothing locked.
*/
-
struct disk_info {
struct md_rdev *rdev, *replacement;
};
#define ALGORITHM_ROTATING_N_RESTART 9 /* DDF PRL=6 RLQ=2 */
#define ALGORITHM_ROTATING_N_CONTINUE 10 /*DDF PRL=6 RLQ=3 */
-
/* For every RAID5 algorithm we define a RAID6 algorithm
* with exactly the same layout for data and parity, and
* with the Q block always on the last device (N-1).
struct mic_device_ctrl __iomem *dc
= (void __iomem *)d + mic_aligned_desc_size(d);
struct mic_vdev *mvdev = (struct mic_vdev *)ioread64(&dc->vdev);
- struct virtio_driver *drv;
if (ioread8(&dc->config_change) != MIC_VIRTIO_PARAM_CONFIG_CHANGED)
return;
dev_dbg(mdrv->dev, "%s %d\n", __func__, __LINE__);
- drv = container_of(mvdev->vdev.dev.driver,
- struct virtio_driver, driver);
- if (drv->config_changed)
- drv->config_changed(&mvdev->vdev);
+ virtio_config_changed(&mvdev->vdev);
iowrite8(1, &dc->guest_ack);
}
blk_queue_prep_rq(mq->queue, mmc_prep_request);
queue_flag_set_unlocked(QUEUE_FLAG_NONROT, mq->queue);
+ queue_flag_clear_unlocked(QUEUE_FLAG_ADD_RANDOM, mq->queue);
if (mmc_can_erase(card))
mmc_queue_setup_discard(mq->queue, card);
continue;
}
+ /*
+ * New (ARM?) devices may have NVRAM in some middle block. Last
+ * block will be checked later, so skip it.
+ */
+ if (offset != master->size - blocksize &&
+ buf[0x000 / 4] == NVRAM_HEADER) {
+ bcm47xxpart_add_part(&parts[curr_part++], "nvram",
+ offset, 0);
+ continue;
+ }
+
/* Read middle of the block */
if (mtd_read(master, offset + 0x8000, 0x4,
&bytes_read, (uint8_t *)buf) < 0) {
udelay(1);
}
+
+ retries--;
}
/* the chip never became ready */
obj-$(CONFIG_MTD_BLOCK2MTD) += block2mtd.o
obj-$(CONFIG_MTD_DATAFLASH) += mtd_dataflash.o
obj-$(CONFIG_MTD_M25P80) += m25p80.o
-obj-$(CONFIG_MTD_NAND_OMAP_BCH) += elm.o
obj-$(CONFIG_MTD_SPEAR_SMI) += spear_smi.o
obj-$(CONFIG_MTD_SST25L) += sst25l.o
obj-$(CONFIG_MTD_BCM47XXSFLASH) += bcm47xxsflash.o
switch (mode) {
case DOC_ASICMODE_RESET:
- pos += seq_printf(s, "reset");
+ pos += seq_puts(s, "reset");
break;
case DOC_ASICMODE_NORMAL:
- pos += seq_printf(s, "normal");
+ pos += seq_puts(s, "normal");
break;
case DOC_ASICMODE_POWERDOWN:
- pos += seq_printf(s, "powerdown");
+ pos += seq_puts(s, "powerdown");
break;
}
- pos += seq_printf(s, ")\n");
+ pos += seq_puts(s, ")\n");
return pos;
}
DEBUGFS_RO_ATTR(asic_mode, dbg_asicmode_show);
pos += seq_printf(s, "Protection = 0x%02x (",
protect);
if (protect & DOC_PROTECT_FOUNDRY_OTP_LOCK)
- pos += seq_printf(s, "FOUNDRY_OTP_LOCK,");
+ pos += seq_puts(s, "FOUNDRY_OTP_LOCK,");
if (protect & DOC_PROTECT_CUSTOMER_OTP_LOCK)
- pos += seq_printf(s, "CUSTOMER_OTP_LOCK,");
+ pos += seq_puts(s, "CUSTOMER_OTP_LOCK,");
if (protect & DOC_PROTECT_LOCK_INPUT)
- pos += seq_printf(s, "LOCK_INPUT,");
+ pos += seq_puts(s, "LOCK_INPUT,");
if (protect & DOC_PROTECT_STICKY_LOCK)
- pos += seq_printf(s, "STICKY_LOCK,");
+ pos += seq_puts(s, "STICKY_LOCK,");
if (protect & DOC_PROTECT_PROTECTION_ENABLED)
- pos += seq_printf(s, "PROTECTION ON,");
+ pos += seq_puts(s, "PROTECTION ON,");
if (protect & DOC_PROTECT_IPL_DOWNLOAD_LOCK)
- pos += seq_printf(s, "IPL_DOWNLOAD_LOCK,");
+ pos += seq_puts(s, "IPL_DOWNLOAD_LOCK,");
if (protect & DOC_PROTECT_PROTECTION_ERROR)
- pos += seq_printf(s, "PROTECT_ERR,");
+ pos += seq_puts(s, "PROTECT_ERR,");
else
- pos += seq_printf(s, "NO_PROTECT_ERR");
- pos += seq_printf(s, ")\n");
+ pos += seq_puts(s, "NO_PROTECT_ERR");
+ pos += seq_puts(s, ")\n");
pos += seq_printf(s, "DPS0 = 0x%02x : "
"Protected area [0x%x - 0x%x] : OTP=%d, READ=%d, "
{
struct mtd_part_parser_data ppdata;
struct flash_platform_data *data;
+ const struct spi_device_id *id = NULL;
struct m25p *flash;
struct spi_nor *nor;
enum read_mode mode = SPI_NOR_NORMAL;
int ret;
+ data = dev_get_platdata(&spi->dev);
+
flash = devm_kzalloc(&spi->dev, sizeof(*flash), GFP_KERNEL);
if (!flash)
return -ENOMEM;
mode = SPI_NOR_QUAD;
else if (spi->mode & SPI_RX_DUAL)
mode = SPI_NOR_DUAL;
- ret = spi_nor_scan(nor, spi_get_device_id(spi), mode);
+
+ if (data && data->name)
+ flash->mtd.name = data->name;
+
+ /* For some (historical?) reason many platforms provide two different
+ * names in flash_platform_data: "name" and "type". Quite often name is
+ * set to "m25p80" and then "type" provides a real chip name.
+ * If that's the case, respect "type" and ignore a "name".
+ */
+ if (data && data->type)
+ id = spi_nor_match_id(data->type);
+
+ /* If we didn't get name from platform, simply use "modalias". */
+ if (!id)
+ id = spi_get_device_id(spi);
+
+ ret = spi_nor_scan(nor, id, mode);
if (ret)
return ret;
- data = dev_get_platdata(&spi->dev);
ppdata.of_node = spi->dev.of_node;
return mtd_device_parse_register(&flash->mtd, NULL, &ppdata,
config MTD_SOLUTIONENGINE
tristate "CFI Flash device mapped on Hitachi SolutionEngine"
- depends on SUPERH && SOLUTION_ENGINE && MTD_CFI && MTD_REDBOOT_PARTS
+ depends on SOLUTION_ENGINE && MTD_CFI && MTD_REDBOOT_PARTS
help
This enables access to the flash chips on the Hitachi SolutionEngine and
similar boards. Say 'Y' if you are building a kernel for such a board.
* @from: flash offset to copy from
* @len: how much to copy
*
- * We rely on the MTD layer to chunk up copies such that a single request here
- * will not cross a window size. This allows us to only wiggle the GPIOs once
- * before falling back to a normal memcpy. Reading the higher layer code shows
- * that this is indeed the case, but add a BUG_ON() to future proof.
+ * The "from" region may straddle more than one window, so toggle the GPIOs for
+ * each window region before reading its data.
*/
static void gf_copy_from(struct map_info *map, void *to, unsigned long from, ssize_t len)
{
struct async_state *state = gf_map_info_to_state(map);
- gf_set_gpios(state, from);
+ int this_len;
- /* BUG if operation crosses the win_size */
- BUG_ON(!((from + len) % state->win_size <= (from + len)));
+ while (len) {
+ if ((from % state->win_size) + len > state->win_size)
+ this_len = state->win_size - (from % state->win_size);
+ else
+ this_len = len;
- /* operation does not cross the win_size, so one shot it */
- memcpy_fromio(to, map->virt + (from % state->win_size), len);
+ gf_set_gpios(state, from);
+ memcpy_fromio(to, map->virt + (from % state->win_size),
+ this_len);
+ len -= this_len;
+ from += this_len;
+ to += this_len;
+ }
}
/**
{
struct async_state *state = gf_map_info_to_state(map);
- gf_set_gpios(state, to);
+ int this_len;
+
+ while (len) {
+ if ((to % state->win_size) + len > state->win_size)
+ this_len = state->win_size - (to % state->win_size);
+ else
+ this_len = len;
- /* BUG if operation crosses the win_size */
- BUG_ON(!((to + len) % state->win_size <= (to + len)));
+ gf_set_gpios(state, to);
+ memcpy_toio(map->virt + (to % state->win_size), from, len);
- /* operation does not cross the win_size, so one shot it */
- memcpy_toio(map->virt + (to % state->win_size), from, len);
+ len -= this_len;
+ to += this_len;
+ from += this_len;
+ }
}
static const char * const part_probe_types[] = {
if (!pcmcia_dev_present(dev->p_dev)) {
pr_debug("device removed\n");
- return 0;
+ return NULL;
}
offset = to & ~(dev->win_size-1);
if (strcmp(of_probe, "ROM") != 0)
dev_warn(&dev->dev, "obsolete_probe: don't know probe "
"type '%s', mapping as rom\n", of_probe);
- return do_map_probe("mtd_rom", map);
+ return do_map_probe("map_rom", map);
}
}
.compatible = "mtd-ram",
.data = (void *)"map_ram",
},
+ {
+ .compatible = "mtd-rom",
+ .data = (void *)"map_rom",
+ },
{
.type = "rom",
.compatible = "direct-mapped"
blk_queue_logical_block_size(new->rq, tr->blksize);
queue_flag_set_unlocked(QUEUE_FLAG_NONROT, new->rq);
+ queue_flag_clear_unlocked(QUEUE_FLAG_ADD_RANDOM, new->rq);
if (tr->discard) {
queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, new->rq);
if (mtd_is_partition(mtd))
return -EINVAL;
+ /* Sanitize user input */
+ p.devname[BLKPG_DEVNAMELTH - 1] = '\0';
+
return mtd_add_partition(mtd, p.devname, p.start, p.length);
case BLKPG_DEL_PARTITION:
*/
static void mtd_release(struct device *dev)
{
- struct mtd_info __maybe_unused *mtd = dev_get_drvdata(dev);
+ struct mtd_info *mtd = dev_get_drvdata(dev);
dev_t index = MTD_DEVT(mtd->index);
- /* remove /dev/mtdXro node if needed */
- if (index)
- device_destroy(&mtd_class, index + 1);
+ /* remove /dev/mtdXro node */
+ device_destroy(&mtd_class, index + 1);
}
static int mtd_cls_suspend(struct device *dev, pm_message_t state)
if (device_register(&mtd->dev) != 0)
goto fail_added;
- if (MTD_DEVT(i))
- device_create(&mtd_class, mtd->dev.parent,
- MTD_DEVT(i) + 1,
- NULL, "mtd%dro", i);
+ device_create(&mtd_class, mtd->dev.parent, MTD_DEVT(i) + 1, NULL,
+ "mtd%dro", i);
pr_debug("mtd: Giving out device %d to %s\n", i, mtd->name);
/* No need to get a refcount on the module containing
*/
int mtd_erase(struct mtd_info *mtd, struct erase_info *instr)
{
- if (instr->addr > mtd->size || instr->len > mtd->size - instr->addr)
+ if (instr->addr >= mtd->size || instr->len > mtd->size - instr->addr)
return -EINVAL;
if (!(mtd->flags & MTD_WRITEABLE))
return -EROFS;
*phys = 0;
if (!mtd->_point)
return -EOPNOTSUPP;
- if (from < 0 || from > mtd->size || len > mtd->size - from)
+ if (from < 0 || from >= mtd->size || len > mtd->size - from)
return -EINVAL;
if (!len)
return 0;
{
if (!mtd->_point)
return -EOPNOTSUPP;
- if (from < 0 || from > mtd->size || len > mtd->size - from)
+ if (from < 0 || from >= mtd->size || len > mtd->size - from)
return -EINVAL;
if (!len)
return 0;
{
if (!mtd->_get_unmapped_area)
return -EOPNOTSUPP;
- if (offset > mtd->size || len > mtd->size - offset)
+ if (offset >= mtd->size || len > mtd->size - offset)
return -EINVAL;
return mtd->_get_unmapped_area(mtd, len, offset, flags);
}
{
int ret_code;
*retlen = 0;
- if (from < 0 || from > mtd->size || len > mtd->size - from)
+ if (from < 0 || from >= mtd->size || len > mtd->size - from)
return -EINVAL;
if (!len)
return 0;
const u_char *buf)
{
*retlen = 0;
- if (to < 0 || to > mtd->size || len > mtd->size - to)
+ if (to < 0 || to >= mtd->size || len > mtd->size - to)
return -EINVAL;
if (!mtd->_write || !(mtd->flags & MTD_WRITEABLE))
return -EROFS;
*retlen = 0;
if (!mtd->_panic_write)
return -EOPNOTSUPP;
- if (to < 0 || to > mtd->size || len > mtd->size - to)
+ if (to < 0 || to >= mtd->size || len > mtd->size - to)
return -EINVAL;
if (!(mtd->flags & MTD_WRITEABLE))
return -EROFS;
{
if (!mtd->_lock)
return -EOPNOTSUPP;
- if (ofs < 0 || ofs > mtd->size || len > mtd->size - ofs)
+ if (ofs < 0 || ofs >= mtd->size || len > mtd->size - ofs)
return -EINVAL;
if (!len)
return 0;
{
if (!mtd->_unlock)
return -EOPNOTSUPP;
- if (ofs < 0 || ofs > mtd->size || len > mtd->size - ofs)
+ if (ofs < 0 || ofs >= mtd->size || len > mtd->size - ofs)
return -EINVAL;
if (!len)
return 0;
{
if (!mtd->_is_locked)
return -EOPNOTSUPP;
- if (ofs < 0 || ofs > mtd->size || len > mtd->size - ofs)
+ if (ofs < 0 || ofs >= mtd->size || len > mtd->size - ofs)
return -EINVAL;
if (!len)
return 0;
int mtd_block_isreserved(struct mtd_info *mtd, loff_t ofs)
{
- if (ofs < 0 || ofs > mtd->size)
+ if (ofs < 0 || ofs >= mtd->size)
return -EINVAL;
if (!mtd->_block_isreserved)
return 0;
int mtd_block_isbad(struct mtd_info *mtd, loff_t ofs)
{
- if (ofs < 0 || ofs > mtd->size)
+ if (ofs < 0 || ofs >= mtd->size)
return -EINVAL;
if (!mtd->_block_isbad)
return 0;
{
if (!mtd->_block_markbad)
return -EOPNOTSUPP;
- if (ofs < 0 || ofs > mtd->size)
+ if (ofs < 0 || ofs >= mtd->size)
return -EINVAL;
if (!(mtd->flags & MTD_WRITEABLE))
return -EROFS;
struct mtdswap_oobdata {
__le16 magic;
__le32 count;
-} __attribute__((packed));
+} __packed;
#define MTDSWAP_MAGIC_CLEAN 0x2095
#define MTDSWAP_MAGIC_DIRTY (MTDSWAP_MAGIC_CLEAN + 1)
seq_printf(s, "total erasures: %lu\n", sum);
- seq_printf(s, "\n");
+ seq_puts(s, "\n");
seq_printf(s, "mtdswap_readsect count: %llu\n", d->sect_read_count);
seq_printf(s, "mtdswap_writesect count: %llu\n", d->sect_write_count);
seq_printf(s, "mtd write count: %llu\n", d->mtd_write_count);
seq_printf(s, "discarded pages count: %llu\n", d->discard_page_count);
- seq_printf(s, "\n");
+ seq_puts(s, "\n");
seq_printf(s, "total pages: %u\n", pages);
seq_printf(s, "pages mapped: %u\n", mapped);
}
eblocks = mtd_div_by_eb(use_size, mtd);
- use_size = eblocks * mtd->erasesize;
+ use_size = (uint64_t)eblocks * mtd->erasesize;
bad_blocks = mtdswap_badblocks(mtd, use_size);
eavailable = eblocks - bad_blocks;
config MTD_NAND_OMAP_BCH
depends on MTD_NAND_OMAP2
- tristate "Support hardware based BCH error correction"
+ bool "Support hardware based BCH error correction"
default n
select BCH
help
locate and correct errors when using BCH ECC scheme. This offloads
the cpu from doing ECC error searching and correction. However some
legacy OMAP families like OMAP2xxx, OMAP3xxx do not have ELM engine
- so they should not enable this config symbol.
+ so this is optional for them.
+
+config MTD_NAND_OMAP_BCH_BUILD
+ def_tristate MTD_NAND_OMAP2 && MTD_NAND_OMAP_BCH
config MTD_NAND_IDS
tristate
obj-$(CONFIG_MTD_NAND_ATMEL) += atmel_nand.o
obj-$(CONFIG_MTD_NAND_GPIO) += gpio.o
obj-$(CONFIG_MTD_NAND_OMAP2) += omap2.o
+obj-$(CONFIG_MTD_NAND_OMAP_BCH_BUILD) += omap_elm.o
obj-$(CONFIG_MTD_NAND_CM_X270) += cmx270_nand.o
obj-$(CONFIG_MTD_NAND_PXA3xx) += pxa3xx_nand.o
obj-$(CONFIG_MTD_NAND_TMIO) += tmio_nand.o
*
*/
+#include <linux/clk.h>
#include <linux/dma-mapping.h>
#include <linux/slab.h>
#include <linux/module.h>
bool use_nfc_sram;
bool write_by_sram;
+ struct clk *clk;
+
bool is_initialized;
struct completion comp_ready;
struct completion comp_cmd_done;
u32 pmecc_lookup_table_offset_512;
u32 pmecc_lookup_table_offset_1024;
- int pmecc_bytes_per_sector;
- int pmecc_sector_number;
int pmecc_degree; /* Degree of remainders */
int pmecc_cw_len; /* Length of codeword */
pos, bit_pos, err_byte, *(buf + byte_pos));
} else {
/* Bit flip in OOB area */
- tmp = sector_num * host->pmecc_bytes_per_sector
+ tmp = sector_num * nand_chip->ecc.bytes
+ (byte_pos - sector_size);
err_byte = ecc[tmp];
ecc[tmp] ^= (1 << bit_pos);
return 0;
normal_check:
- for (i = 0; i < host->pmecc_sector_number; i++) {
+ for (i = 0; i < nand_chip->ecc.steps; i++) {
err_nbr = 0;
if (pmecc_stat & 0x1) {
buf_pos = buf + i * host->pmecc_sector_size;
return -EIO;
} else {
pmecc_correct_data(mtd, buf_pos, ecc, i,
- host->pmecc_bytes_per_sector, err_nbr);
+ nand_chip->ecc.bytes, err_nbr);
mtd->ecc_stats.corrected += err_nbr;
total_err += err_nbr;
}
cpu_relax();
}
- for (i = 0; i < host->pmecc_sector_number; i++) {
- for (j = 0; j < host->pmecc_bytes_per_sector; j++) {
+ for (i = 0; i < chip->ecc.steps; i++) {
+ for (j = 0; j < chip->ecc.bytes; j++) {
int pos;
- pos = i * host->pmecc_bytes_per_sector + j;
+ pos = i * chip->ecc.bytes + j;
chip->oob_poi[eccpos[pos]] =
pmecc_readb_ecc_relaxed(host->ecc, i, j);
}
else if (host->pmecc_sector_size == 1024)
val |= PMECC_CFG_SECTOR1024;
- switch (host->pmecc_sector_number) {
+ switch (nand_chip->ecc.steps) {
case 1:
val |= PMECC_CFG_PAGE_1SECTOR;
break;
host->ecc = devm_ioremap_resource(&pdev->dev, regs);
if (IS_ERR(host->ecc)) {
- dev_err(host->dev, "ioremap failed\n");
err_no = PTR_ERR(host->ecc);
goto err;
}
regs_pmerr = platform_get_resource(pdev, IORESOURCE_MEM, 2);
host->pmerrloc_base = devm_ioremap_resource(&pdev->dev, regs_pmerr);
if (IS_ERR(host->pmerrloc_base)) {
- dev_err(host->dev,
- "Can not get I/O resource for PMECC ERRLOC controller!\n");
err_no = PTR_ERR(host->pmerrloc_base);
goto err;
}
regs_rom = platform_get_resource(pdev, IORESOURCE_MEM, 3);
host->pmecc_rom_base = devm_ioremap_resource(&pdev->dev, regs_rom);
if (IS_ERR(host->pmecc_rom_base)) {
- dev_err(host->dev, "Can not get I/O resource for ROM!\n");
err_no = PTR_ERR(host->pmecc_rom_base);
goto err;
}
/* set ECC page size and oob layout */
switch (mtd->writesize) {
+ case 512:
+ case 1024:
case 2048:
+ case 4096:
+ case 8192:
+ if (sector_size > mtd->writesize) {
+ dev_err(host->dev, "pmecc sector size is bigger than the page size!\n");
+ err_no = -EINVAL;
+ goto err;
+ }
+
host->pmecc_degree = (sector_size == 512) ?
PMECC_GF_DIMENSION_13 : PMECC_GF_DIMENSION_14;
host->pmecc_cw_len = (1 << host->pmecc_degree) - 1;
- host->pmecc_sector_number = mtd->writesize / sector_size;
- host->pmecc_bytes_per_sector = pmecc_get_ecc_bytes(
- cap, sector_size);
host->pmecc_alpha_to = pmecc_get_alpha_to(host);
host->pmecc_index_of = host->pmecc_rom_base +
host->pmecc_lookup_table_offset;
- nand_chip->ecc.steps = host->pmecc_sector_number;
nand_chip->ecc.strength = cap;
- nand_chip->ecc.bytes = host->pmecc_bytes_per_sector;
- nand_chip->ecc.total = host->pmecc_bytes_per_sector *
- host->pmecc_sector_number;
+ nand_chip->ecc.bytes = pmecc_get_ecc_bytes(cap, sector_size);
+ nand_chip->ecc.steps = mtd->writesize / sector_size;
+ nand_chip->ecc.total = nand_chip->ecc.bytes *
+ nand_chip->ecc.steps;
if (nand_chip->ecc.total > mtd->oobsize - 2) {
dev_err(host->dev, "No room for ECC bytes\n");
err_no = -EINVAL;
nand_chip->ecc.layout = &atmel_pmecc_oobinfo;
break;
- case 512:
- case 1024:
- case 4096:
- /* TODO */
+ default:
dev_warn(host->dev,
"Unsupported page size for PMECC, use Software ECC\n");
- default:
/* page size not handled by HW ECC */
/* switching back to soft ECC */
nand_chip->ecc.mode = NAND_ECC_SOFT;
}
host->ecc = devm_ioremap_resource(&pdev->dev, regs);
- if (IS_ERR(host->ecc)) {
- dev_err(host->dev, "ioremap failed\n");
+ if (IS_ERR(host->ecc))
return PTR_ERR(host->ecc);
- }
/* ECC is calculated for the whole page (1 step) */
nand_chip->ecc.size = mtd->writesize;
if (offset || (data_len < mtd->writesize))
return -EINVAL;
- cfg = nfc_readl(host->nfc->hsmc_regs, CFG);
len = mtd->writesize;
-
- if (unlikely(raw)) {
- len += mtd->oobsize;
- nfc_writel(host->nfc->hsmc_regs, CFG, cfg | NFC_CFG_WSPARE);
- } else
- nfc_writel(host->nfc->hsmc_regs, CFG, cfg & ~NFC_CFG_WSPARE);
-
/* Copy page data to sram that will write to nand via NFC */
if (use_dma) {
if (atmel_nand_dma_op(mtd, (void *)buf, len, 0) != 0)
memcpy32_toio(sram, buf, len);
}
+ cfg = nfc_readl(host->nfc->hsmc_regs, CFG);
+ if (unlikely(raw) && oob_required) {
+ memcpy32_toio(sram + len, chip->oob_poi, mtd->oobsize);
+ len += mtd->oobsize;
+ nfc_writel(host->nfc->hsmc_regs, CFG, cfg | NFC_CFG_WSPARE);
+ } else {
+ nfc_writel(host->nfc->hsmc_regs, CFG, cfg & ~NFC_CFG_WSPARE);
+ }
+
if (chip->ecc.mode == NAND_ECC_HW && host->has_pmecc)
/*
* When use NFC sram, need set up PMECC before send
mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
host->io_base = devm_ioremap_resource(&pdev->dev, mem);
if (IS_ERR(host->io_base)) {
- dev_err(&pdev->dev, "atmel_nand: ioremap resource failed\n");
res = PTR_ERR(host->io_base);
goto err_nand_ioremap;
}
}
nand_chip->ecc.mode = host->board.ecc_mode;
- nand_chip->chip_delay = 20; /* 20us command delay time */
+ nand_chip->chip_delay = 40; /* 40us command delay time */
if (host->board.bus_width_16) /* 16-bit bus width */
nand_chip->options |= NAND_BUSWIDTH_16;
{
struct atmel_nfc *nfc = &nand_nfc;
struct resource *nfc_cmd_regs, *nfc_hsmc_regs, *nfc_sram;
+ int ret;
nfc_cmd_regs = platform_get_resource(pdev, IORESOURCE_MEM, 0);
nfc->base_cmd_regs = devm_ioremap_resource(&pdev->dev, nfc_cmd_regs);
nfc_writel(nfc->hsmc_regs, IDR, 0xffffffff);
nfc_readl(nfc->hsmc_regs, SR); /* clear the NFC_SR */
+ nfc->clk = devm_clk_get(&pdev->dev, NULL);
+ if (!IS_ERR(nfc->clk)) {
+ ret = clk_prepare_enable(nfc->clk);
+ if (ret)
+ return ret;
+ } else {
+ dev_warn(&pdev->dev, "NFC clock missing, update your Device Tree");
+ }
+
nfc->is_initialized = true;
dev_info(&pdev->dev, "NFC is probed.\n");
+
+ return 0;
+}
+
+static int atmel_nand_nfc_remove(struct platform_device *pdev)
+{
+ struct atmel_nfc *nfc = &nand_nfc;
+
+ if (!IS_ERR(nfc->clk))
+ clk_disable_unprepare(nfc->clk);
+
return 0;
}
.of_match_table = of_match_ptr(atmel_nand_nfc_match),
},
.probe = atmel_nand_nfc_probe,
+ .remove = atmel_nand_nfc_remove,
};
static struct platform_driver atmel_nand_driver = {
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/slab.h>
+#include <linux/delay.h>
#include <linux/bcma/bcma.h>
/* Broadcom uses 1'000'000 but it seems to be too many. Tests on WNDR4500 has
#define NFLASH_SECTOR_SIZE 512
#define NCTL_CMD0 0x00010000
+#define NCTL_COL 0x00020000 /* Update column with value from BCMA_CC_NFLASH_COL_ADDR */
+#define NCTL_ROW 0x00040000 /* Update row (page) with value from BCMA_CC_NFLASH_ROW_ADDR */
#define NCTL_CMD1W 0x00080000
#define NCTL_READ 0x00100000
#define NCTL_WRITE 0x00200000
b47n->curr_page_addr);
/* Prepare to read */
- ctlcode = NCTL_CSA | NCTL_CMD1W | 0x00040000 | 0x00020000 |
+ ctlcode = NCTL_CSA | NCTL_CMD1W | NCTL_ROW | NCTL_COL |
NCTL_CMD0;
ctlcode |= NAND_CMD_READSTART << 8;
if (bcm47xxnflash_ops_bcm4706_ctl_cmd(b47n->cc, ctlcode))
* NAND chip ops
**************************************************/
+static void bcm47xxnflash_ops_bcm4706_cmd_ctrl(struct mtd_info *mtd, int cmd,
+ unsigned int ctrl)
+{
+ struct nand_chip *nand_chip = (struct nand_chip *)mtd->priv;
+ struct bcm47xxnflash *b47n = (struct bcm47xxnflash *)nand_chip->priv;
+ u32 code = 0;
+
+ if (cmd == NAND_CMD_NONE)
+ return;
+
+ if (cmd & NAND_CTRL_CLE)
+ code = cmd | NCTL_CMD0;
+
+ /* nCS is not needed for reset command */
+ if (cmd != NAND_CMD_RESET)
+ code |= NCTL_CSA;
+
+ bcm47xxnflash_ops_bcm4706_ctl_cmd(b47n->cc, code);
+}
+
/* Default nand_select_chip calls cmd_ctrl, which is not used in BCM4706 */
static void bcm47xxnflash_ops_bcm4706_select_chip(struct mtd_info *mtd,
int chip)
return;
}
+static int bcm47xxnflash_ops_bcm4706_dev_ready(struct mtd_info *mtd)
+{
+ struct nand_chip *nand_chip = (struct nand_chip *)mtd->priv;
+ struct bcm47xxnflash *b47n = (struct bcm47xxnflash *)nand_chip->priv;
+
+ return !!(bcma_cc_read32(b47n->cc, BCMA_CC_NFLASH_CTL) & NCTL_READY);
+}
+
/*
* Default nand_command and nand_command_lp don't match BCM4706 hardware layout.
* For example, reading chip id is performed in a non-standard way.
switch (command) {
case NAND_CMD_RESET:
- pr_warn("Chip reset not implemented yet\n");
+ nand_chip->cmd_ctrl(mtd, command, NAND_CTRL_CLE);
+
+ ndelay(100);
+ nand_wait_ready(mtd);
break;
case NAND_CMD_READID:
ctlcode = NCTL_CSA | 0x01000000 | NCTL_CMD1W | NCTL_CMD0;
case NAND_CMD_ERASE1:
bcma_cc_write32(cc, BCMA_CC_NFLASH_ROW_ADDR,
b47n->curr_page_addr);
- ctlcode = 0x00040000 | NCTL_CMD1W | NCTL_CMD0 |
+ ctlcode = NCTL_ROW | NCTL_CMD1W | NCTL_CMD0 |
NAND_CMD_ERASE1 | (NAND_CMD_ERASE2 << 8);
if (bcm47xxnflash_ops_bcm4706_ctl_cmd(cc, ctlcode))
pr_err("ERASE1 failed\n");
b47n->curr_page_addr);
/* Prepare to write */
- ctlcode = 0x40000000 | 0x00040000 | 0x00020000 | 0x00010000;
+ ctlcode = 0x40000000 | NCTL_ROW | NCTL_COL | NCTL_CMD0;
ctlcode |= NAND_CMD_SEQIN;
if (bcm47xxnflash_ops_bcm4706_ctl_cmd(cc, ctlcode))
pr_err("SEQIN failed\n");
break;
case NAND_CMD_PAGEPROG:
- if (bcm47xxnflash_ops_bcm4706_ctl_cmd(cc, 0x00010000 |
+ if (bcm47xxnflash_ops_bcm4706_ctl_cmd(cc, NCTL_CMD0 |
NAND_CMD_PAGEPROG))
pr_err("PAGEPROG failed\n");
if (bcm47xxnflash_ops_bcm4706_poll(cc))
int bcm47xxnflash_ops_bcm4706_init(struct bcm47xxnflash *b47n)
{
+ struct nand_chip *nand_chip = (struct nand_chip *)&b47n->nand_chip;
int err;
u32 freq;
u16 clock;
u32 val;
b47n->nand_chip.select_chip = bcm47xxnflash_ops_bcm4706_select_chip;
+ nand_chip->cmd_ctrl = bcm47xxnflash_ops_bcm4706_cmd_ctrl;
+ nand_chip->dev_ready = bcm47xxnflash_ops_bcm4706_dev_ready;
b47n->nand_chip.cmdfunc = bcm47xxnflash_ops_bcm4706_cmdfunc;
b47n->nand_chip.read_byte = bcm47xxnflash_ops_bcm4706_read_byte;
b47n->nand_chip.read_buf = bcm47xxnflash_ops_bcm4706_read_buf;
b47n->nand_chip.write_buf = bcm47xxnflash_ops_bcm4706_write_buf;
+
+ nand_chip->chip_delay = 50;
b47n->nand_chip.bbt_options = NAND_BBT_USE_FLASH;
b47n->nand_chip.ecc.mode = NAND_ECC_NONE; /* TODO: implement ECC */
/* Configure wait counters */
if (b47n->cc->status & BCMA_CC_CHIPST_4706_PKG_OPTION) {
- freq = 100000000;
+ /* 400 MHz */
+ freq = 400000000 / 4;
} else {
freq = bcma_chipco_pll_read(b47n->cc, 4);
- freq = (freq * 0xFFF) >> 3;
- freq = (freq * 25000000) >> 3;
+ freq = (freq & 0xFFF) >> 3;
+ /* Fixed reference clock 25 MHz and m = 2 */
+ freq = (freq * 25000000 / 2) / 4;
}
clock = freq / 1000000;
w0 = bcm47xxnflash_ops_bcm4706_ns_to_cycle(15, clock);
MODULE_LICENSE("GPL");
-/* We define a module parameter that allows the user to override
+/*
+ * We define a module parameter that allows the user to override
* the hardware and decide what timing mode should be used.
*/
#define NAND_DEFAULT_TIMINGS -1
static int onfi_timing_mode = NAND_DEFAULT_TIMINGS;
module_param(onfi_timing_mode, int, S_IRUGO);
-MODULE_PARM_DESC(onfi_timing_mode, "Overrides default ONFI setting."
- " -1 indicates use default timings");
+MODULE_PARM_DESC(onfi_timing_mode,
+ "Overrides default ONFI setting. -1 indicates use default timings");
#define DENALI_NAND_NAME "denali-nand"
-/* We define a macro here that combines all interrupts this driver uses into
- * a single constant value, for convenience. */
+/*
+ * We define a macro here that combines all interrupts this driver uses into
+ * a single constant value, for convenience.
+ */
#define DENALI_IRQ_ALL (INTR_STATUS__DMA_CMD_COMP | \
INTR_STATUS__ECC_TRANSACTION_DONE | \
INTR_STATUS__ECC_ERR | \
INTR_STATUS__RST_COMP | \
INTR_STATUS__ERASE_COMP)
-/* indicates whether or not the internal value for the flash bank is
- * valid or not */
+/*
+ * indicates whether or not the internal value for the flash bank is
+ * valid or not
+ */
#define CHIP_SELECT_INVALID -1
#define SUPPORT_8BITECC 1
-/* This macro divides two integers and rounds fractional values up
- * to the nearest integer value. */
+/*
+ * This macro divides two integers and rounds fractional values up
+ * to the nearest integer value.
+ */
#define CEIL_DIV(X, Y) (((X)%(Y)) ? ((X)/(Y)+1) : ((X)/(Y)))
-/* this macro allows us to convert from an MTD structure to our own
+/*
+ * this macro allows us to convert from an MTD structure to our own
* device context (denali) structure.
*/
#define mtd_to_denali(m) container_of(m, struct denali_nand_info, mtd)
-/* These constants are defined by the driver to enable common driver
- * configuration options. */
+/*
+ * These constants are defined by the driver to enable common driver
+ * configuration options.
+ */
#define SPARE_ACCESS 0x41
#define MAIN_ACCESS 0x42
#define MAIN_SPARE_ACCESS 0x43
+#define PIPELINE_ACCESS 0x2000
#define DENALI_READ 0
#define DENALI_WRITE 0x100
#define ADDR_CYCLE 1
#define STATUS_CYCLE 2
-/* this is a helper macro that allows us to
- * format the bank into the proper bits for the controller */
+/*
+ * this is a helper macro that allows us to
+ * format the bank into the proper bits for the controller
+ */
#define BANK(x) ((x) << 24)
/* forward declarations */
uint32_t int_mask);
static uint32_t read_interrupt_status(struct denali_nand_info *denali);
-/* Certain operations for the denali NAND controller use
- * an indexed mode to read/write data. The operation is
- * performed by writing the address value of the command
- * to the device memory followed by the data. This function
+/*
+ * Certain operations for the denali NAND controller use an indexed mode to
+ * read/write data. The operation is performed by writing the address value
+ * of the command to the device memory followed by the data. This function
* abstracts this common operation.
-*/
+ */
static void index_addr(struct denali_nand_info *denali,
uint32_t address, uint32_t data)
{
*pdata = ioread32(denali->flash_mem + 0x10);
}
-/* We need to buffer some data for some of the NAND core routines.
- * The operations manage buffering that data. */
+/*
+ * We need to buffer some data for some of the NAND core routines.
+ * The operations manage buffering that data.
+ */
static void reset_buf(struct denali_nand_info *denali)
{
denali->buf.head = denali->buf.tail = 0;
/* reads the status of the device */
static void read_status(struct denali_nand_info *denali)
{
- uint32_t cmd = 0x0;
+ uint32_t cmd;
/* initialize the data buffer to store status */
reset_buf(denali);
/* resets a specific device connected to the core */
static void reset_bank(struct denali_nand_info *denali)
{
- uint32_t irq_status = 0;
- uint32_t irq_mask = INTR_STATUS__RST_COMP |
- INTR_STATUS__TIME_OUT;
+ uint32_t irq_status;
+ uint32_t irq_mask = INTR_STATUS__RST_COMP | INTR_STATUS__TIME_OUT;
clear_interrupts(denali);
/* Reset the flash controller */
static uint16_t denali_nand_reset(struct denali_nand_info *denali)
{
- uint32_t i;
+ int i;
dev_dbg(denali->dev, "%s, Line %d, Function: %s\n",
- __FILE__, __LINE__, __func__);
+ __FILE__, __LINE__, __func__);
- for (i = 0 ; i < denali->max_banks; i++)
+ for (i = 0; i < denali->max_banks; i++)
iowrite32(INTR_STATUS__RST_COMP | INTR_STATUS__TIME_OUT,
denali->flash_reg + INTR_STATUS(i));
- for (i = 0 ; i < denali->max_banks; i++) {
+ for (i = 0; i < denali->max_banks; i++) {
iowrite32(1 << i, denali->flash_reg + DEVICE_RESET);
- while (!(ioread32(denali->flash_reg +
- INTR_STATUS(i)) &
+ while (!(ioread32(denali->flash_reg + INTR_STATUS(i)) &
(INTR_STATUS__RST_COMP | INTR_STATUS__TIME_OUT)))
cpu_relax();
if (ioread32(denali->flash_reg + INTR_STATUS(i)) &
for (i = 0; i < denali->max_banks; i++)
iowrite32(INTR_STATUS__RST_COMP | INTR_STATUS__TIME_OUT,
- denali->flash_reg + INTR_STATUS(i));
+ denali->flash_reg + INTR_STATUS(i));
return PASS;
}
-/* this routine calculates the ONFI timing values for a given mode and
+/*
+ * this routine calculates the ONFI timing values for a given mode and
* programs the clocking register accordingly. The mode is determined by
* the get_onfi_nand_para routine.
*/
uint16_t addr_2_data, re_2_we, re_2_re, we_2_re, cs_cnt;
dev_dbg(denali->dev, "%s, Line %d, Function: %s\n",
- __FILE__, __LINE__, __func__);
+ __FILE__, __LINE__, __func__);
en_lo = CEIL_DIV(Trp[mode], CLK_X);
en_hi = CEIL_DIV(Treh[mode], CLK_X);
data_invalid_rloh = (en_lo + en_hi) * CLK_X + Trloh[mode];
- data_invalid =
- data_invalid_rhoh <
- data_invalid_rloh ? data_invalid_rhoh : data_invalid_rloh;
+ data_invalid = data_invalid_rhoh < data_invalid_rloh ?
+ data_invalid_rhoh : data_invalid_rloh;
dv_window = data_invalid - Trea[mode];
acc_clks = CEIL_DIV(Trea[mode], CLK_X);
- while (((acc_clks * CLK_X) - Trea[mode]) < 3)
+ while (acc_clks * CLK_X - Trea[mode] < 3)
acc_clks++;
- if ((data_invalid - acc_clks * CLK_X) < 2)
+ if (data_invalid - acc_clks * CLK_X < 2)
dev_warn(denali->dev, "%s, Line %d: Warning!\n",
- __FILE__, __LINE__);
+ __FILE__, __LINE__);
addr_2_data = CEIL_DIV(Tadl[mode], CLK_X);
re_2_we = CEIL_DIV(Trhw[mode], CLK_X);
cs_cnt = 1;
if (Tcea[mode]) {
- while (((cs_cnt * CLK_X) + Trea[mode]) < Tcea[mode])
+ while (cs_cnt * CLK_X + Trea[mode] < Tcea[mode])
cs_cnt++;
}
#endif
/* Sighting 3462430: Temporary hack for MT29F128G08CJABAWP:B */
- if ((ioread32(denali->flash_reg + MANUFACTURER_ID) == 0) &&
- (ioread32(denali->flash_reg + DEVICE_ID) == 0x88))
+ if (ioread32(denali->flash_reg + MANUFACTURER_ID) == 0 &&
+ ioread32(denali->flash_reg + DEVICE_ID) == 0x88)
acc_clks = 6;
iowrite32(acc_clks, denali->flash_reg + ACC_CLKS);
static uint16_t get_onfi_nand_para(struct denali_nand_info *denali)
{
int i;
- /* we needn't to do a reset here because driver has already
+
+ /*
+ * we needn't to do a reset here because driver has already
* reset all the banks before
- * */
+ */
if (!(ioread32(denali->flash_reg + ONFI_TIMING_MODE) &
ONFI_TIMING_MODE__VALUE))
return FAIL;
nand_onfi_timing_set(denali, i);
- /* By now, all the ONFI devices we know support the page cache */
- /* rw feature. So here we enable the pipeline_rw_ahead feature */
+ /*
+ * By now, all the ONFI devices we know support the page cache
+ * rw feature. So here we enable the pipeline_rw_ahead feature
+ */
/* iowrite32(1, denali->flash_reg + CACHE_WRITE_ENABLE); */
/* iowrite32(1, denali->flash_reg + CACHE_READ_ENABLE); */
{
uint32_t tmp;
- /* Workaround to fix a controller bug which reports a wrong */
- /* spare area size for some kind of Toshiba NAND device */
+ /*
+ * Workaround to fix a controller bug which reports a wrong
+ * spare area size for some kind of Toshiba NAND device
+ */
if ((ioread32(denali->flash_reg + DEVICE_MAIN_AREA_SIZE) == 4096) &&
(ioread32(denali->flash_reg + DEVICE_SPARE_AREA_SIZE) == 64)) {
iowrite32(216, denali->flash_reg + DEVICE_SPARE_AREA_SIZE);
break;
default:
dev_warn(denali->dev,
- "Spectra: Unknown Hynix NAND (Device ID: 0x%x)."
- "Will use default parameter values instead.\n",
- device_id);
+ "Spectra: Unknown Hynix NAND (Device ID: 0x%x).\n"
+ "Will use default parameter values instead.\n",
+ device_id);
}
}
-/* determines how many NAND chips are connected to the controller. Note for
+/*
+ * determines how many NAND chips are connected to the controller. Note for
* Intel CE4100 devices we don't support more than one device.
*/
static void find_valid_banks(struct denali_nand_info *denali)
denali->total_used_banks = 1;
for (i = 0; i < denali->max_banks; i++) {
- index_addr(denali, (uint32_t)(MODE_11 | (i << 24) | 0), 0x90);
- index_addr(denali, (uint32_t)(MODE_11 | (i << 24) | 1), 0);
- index_addr_read_data(denali,
- (uint32_t)(MODE_11 | (i << 24) | 2), &id[i]);
+ index_addr(denali, MODE_11 | (i << 24) | 0, 0x90);
+ index_addr(denali, MODE_11 | (i << 24) | 1, 0);
+ index_addr_read_data(denali, MODE_11 | (i << 24) | 2, &id[i]);
dev_dbg(denali->dev,
"Return 1st ID for bank[%d]: %x\n", i, id[i]);
}
if (denali->platform == INTEL_CE4100) {
- /* Platform limitations of the CE4100 device limit
+ /*
+ * Platform limitations of the CE4100 device limit
* users to a single chip solution for NAND.
* Multichip support is not enabled.
*/
if (denali->total_used_banks != 1) {
dev_err(denali->dev,
- "Sorry, Intel CE4100 only supports "
- "a single NAND device.\n");
+ "Sorry, Intel CE4100 only supports a single NAND device.\n");
BUG();
}
}
static void detect_partition_feature(struct denali_nand_info *denali)
{
- /* For MRST platform, denali->fwblks represent the
+ /*
+ * For MRST platform, denali->fwblks represent the
* number of blocks firmware is taken,
* FW is in protect partition and MTD driver has no
* permission to access it. So let driver know how many
* blocks it can't touch.
- * */
+ */
if (ioread32(denali->flash_reg + FEATURES) & FEATURES__PARTITION) {
if ((ioread32(denali->flash_reg + PERM_SRC_ID(1)) &
PERM_SRC_ID__SRCID) == SPECTRA_PARTITION_ID) {
+
(ioread32(denali->flash_reg + MIN_BLK_ADDR(1)) &
MIN_BLK_ADDR__VALUE);
- } else
+ } else {
denali->fwblks = SPECTRA_START_BLOCK;
- } else
+ }
+ } else {
denali->fwblks = SPECTRA_START_BLOCK;
+ }
}
static uint16_t denali_nand_timing_set(struct denali_nand_info *denali)
{
uint16_t status = PASS;
uint32_t id_bytes[8], addr;
- uint8_t i, maf_id, device_id;
+ uint8_t maf_id, device_id;
+ int i;
- dev_dbg(denali->dev,
- "%s, Line %d, Function: %s\n",
+ dev_dbg(denali->dev, "%s, Line %d, Function: %s\n",
__FILE__, __LINE__, __func__);
- /* Use read id method to get device ID and other
- * params. For some NAND chips, controller can't
- * report the correct device ID by reading from
- * DEVICE_ID register
- * */
- addr = (uint32_t)MODE_11 | BANK(denali->flash_bank);
- index_addr(denali, (uint32_t)addr | 0, 0x90);
- index_addr(denali, (uint32_t)addr | 1, 0);
+ /*
+ * Use read id method to get device ID and other params.
+ * For some NAND chips, controller can't report the correct
+ * device ID by reading from DEVICE_ID register
+ */
+ addr = MODE_11 | BANK(denali->flash_bank);
+ index_addr(denali, addr | 0, 0x90);
+ index_addr(denali, addr | 1, 0);
for (i = 0; i < 8; i++)
index_addr_read_data(denali, addr | 2, &id_bytes[i]);
maf_id = id_bytes[0];
}
dev_info(denali->dev,
- "Dump timing register values:"
+ "Dump timing register values:\n"
"acc_clks: %d, re_2_we: %d, re_2_re: %d\n"
"we_2_re: %d, addr_2_data: %d, rdwr_en_lo_cnt: %d\n"
"rdwr_en_hi_cnt: %d, cs_setup_cnt: %d\n",
detect_partition_feature(denali);
- /* If the user specified to override the default timings
+ /*
+ * If the user specified to override the default timings
* with a specific ONFI mode, we apply those changes here.
*/
if (onfi_timing_mode != NAND_DEFAULT_TIMINGS)
uint16_t INT_ENABLE)
{
dev_dbg(denali->dev, "%s, Line %d, Function: %s\n",
- __FILE__, __LINE__, __func__);
+ __FILE__, __LINE__, __func__);
if (INT_ENABLE)
iowrite32(1, denali->flash_reg + GLOBAL_INT_ENABLE);
iowrite32(0, denali->flash_reg + GLOBAL_INT_ENABLE);
}
-/* validation function to verify that the controlling software is making
+/*
+ * validation function to verify that the controlling software is making
* a valid request
*/
static inline bool is_flash_bank_valid(int flash_bank)
{
- return (flash_bank >= 0 && flash_bank < 4);
+ return flash_bank >= 0 && flash_bank < 4;
}
static void denali_irq_init(struct denali_nand_info *denali)
{
- uint32_t int_mask = 0;
+ uint32_t int_mask;
int i;
/* Disable global interrupts */
iowrite32(int_mask, denali->flash_reg + INTR_EN(i));
}
-/* This function only returns when an interrupt that this driver cares about
+/*
+ * This function only returns when an interrupt that this driver cares about
* occurs. This is to reduce the overhead of servicing interrupts
*/
static inline uint32_t denali_irq_detected(struct denali_nand_info *denali)
static inline void clear_interrupt(struct denali_nand_info *denali,
uint32_t irq_mask)
{
- uint32_t intr_status_reg = 0;
+ uint32_t intr_status_reg;
intr_status_reg = INTR_STATUS(denali->flash_bank);
static void clear_interrupts(struct denali_nand_info *denali)
{
- uint32_t status = 0x0;
+ uint32_t status;
+
spin_lock_irq(&denali->irq_lock);
status = read_interrupt_status(denali);
static uint32_t read_interrupt_status(struct denali_nand_info *denali)
{
- uint32_t intr_status_reg = 0;
+ uint32_t intr_status_reg;
intr_status_reg = INTR_STATUS(denali->flash_bank);
return ioread32(denali->flash_reg + intr_status_reg);
}
-/* This is the interrupt service routine. It handles all interrupts
- * sent to this device. Note that on CE4100, this is a shared
- * interrupt.
+/*
+ * This is the interrupt service routine. It handles all interrupts
+ * sent to this device. Note that on CE4100, this is a shared interrupt.
*/
static irqreturn_t denali_isr(int irq, void *dev_id)
{
struct denali_nand_info *denali = dev_id;
- uint32_t irq_status = 0x0;
+ uint32_t irq_status;
irqreturn_t result = IRQ_NONE;
spin_lock(&denali->irq_lock);
- /* check to see if a valid NAND chip has
- * been selected.
- */
+ /* check to see if a valid NAND chip has been selected. */
if (is_flash_bank_valid(denali->flash_bank)) {
- /* check to see if controller generated
- * the interrupt, since this is a shared interrupt */
+ /*
+ * check to see if controller generated the interrupt,
+ * since this is a shared interrupt
+ */
irq_status = denali_irq_detected(denali);
if (irq_status != 0) {
/* handle interrupt */
/* first acknowledge it */
clear_interrupt(denali, irq_status);
- /* store the status in the device context for someone
- to read */
+ /*
+ * store the status in the device context for someone
+ * to read
+ */
denali->irq_status |= irq_status;
/* notify anyone who cares that it happened */
complete(&denali->complete);
static uint32_t wait_for_irq(struct denali_nand_info *denali, uint32_t irq_mask)
{
- unsigned long comp_res = 0;
- uint32_t intr_status = 0;
- bool retry = false;
+ unsigned long comp_res;
+ uint32_t intr_status;
unsigned long timeout = msecs_to_jiffies(1000);
do {
spin_unlock_irq(&denali->irq_lock);
/* our interrupt was detected */
break;
- } else {
- /* these are not the interrupts you are looking for -
- * need to wait again */
- spin_unlock_irq(&denali->irq_lock);
- retry = true;
}
+
+ /*
+ * these are not the interrupts you are looking for -
+ * need to wait again
+ */
+ spin_unlock_irq(&denali->irq_lock);
} while (comp_res != 0);
if (comp_res == 0) {
return intr_status;
}
-/* This helper function setups the registers for ECC and whether or not
- * the spare area will be transferred. */
+/*
+ * This helper function setups the registers for ECC and whether or not
+ * the spare area will be transferred.
+ */
static void setup_ecc_for_xfer(struct denali_nand_info *denali, bool ecc_en,
bool transfer_spare)
{
- int ecc_en_flag = 0, transfer_spare_flag = 0;
+ int ecc_en_flag, transfer_spare_flag;
/* set ECC, transfer spare bits if needed */
ecc_en_flag = ecc_en ? ECC_ENABLE__FLAG : 0;
/* Enable spare area/ECC per user's request. */
iowrite32(ecc_en_flag, denali->flash_reg + ECC_ENABLE);
- iowrite32(transfer_spare_flag,
- denali->flash_reg + TRANSFER_SPARE_REG);
+ iowrite32(transfer_spare_flag, denali->flash_reg + TRANSFER_SPARE_REG);
}
-/* sends a pipeline command operation to the controller. See the Denali NAND
+/*
+ * sends a pipeline command operation to the controller. See the Denali NAND
* controller's user guide for more information (section 4.2.3.6).
*/
static int denali_send_pipeline_cmd(struct denali_nand_info *denali,
- bool ecc_en,
- bool transfer_spare,
- int access_type,
- int op)
+ bool ecc_en, bool transfer_spare,
+ int access_type, int op)
{
int status = PASS;
- uint32_t addr = 0x0, cmd = 0x0, page_count = 1, irq_status = 0,
- irq_mask = 0;
+ uint32_t page_count = 1;
+ uint32_t addr, cmd, irq_status, irq_mask;
if (op == DENALI_READ)
irq_mask = INTR_STATUS__LOAD_COMP;
setup_ecc_for_xfer(denali, ecc_en, transfer_spare);
- /* clear interrupts */
clear_interrupts(denali);
addr = BANK(denali->flash_bank) | denali->page;
} else if (op == DENALI_WRITE && access_type == SPARE_ACCESS) {
/* read spare area */
cmd = MODE_10 | addr;
- index_addr(denali, (uint32_t)cmd, access_type);
+ index_addr(denali, cmd, access_type);
cmd = MODE_01 | addr;
iowrite32(cmd, denali->flash_mem);
} else if (op == DENALI_READ) {
/* setup page read request for access type */
cmd = MODE_10 | addr;
- index_addr(denali, (uint32_t)cmd, access_type);
+ index_addr(denali, cmd, access_type);
- /* page 33 of the NAND controller spec indicates we should not
- use the pipeline commands in Spare area only mode. So we
- don't.
+ /*
+ * page 33 of the NAND controller spec indicates we should not
+ * use the pipeline commands in Spare area only mode.
+ * So we don't.
*/
if (access_type == SPARE_ACCESS) {
cmd = MODE_01 | addr;
iowrite32(cmd, denali->flash_mem);
} else {
- index_addr(denali, (uint32_t)cmd,
- 0x2000 | op | page_count);
+ index_addr(denali, cmd,
+ PIPELINE_ACCESS | op | page_count);
- /* wait for command to be accepted
+ /*
+ * wait for command to be accepted
* can always use status0 bit as the
- * mask is identical for each
- * bank. */
+ * mask is identical for each bank.
+ */
irq_status = wait_for_irq(denali, irq_mask);
if (irq_status == 0) {
dev_err(denali->dev,
- "cmd, page, addr on timeout "
- "(0x%x, 0x%x, 0x%x)\n",
- cmd, denali->page, addr);
+ "cmd, page, addr on timeout (0x%x, 0x%x, 0x%x)\n",
+ cmd, denali->page, addr);
status = FAIL;
} else {
cmd = MODE_01 | addr;
/* helper function that simply writes a buffer to the flash */
static int write_data_to_flash_mem(struct denali_nand_info *denali,
- const uint8_t *buf,
- int len)
+ const uint8_t *buf, int len)
{
- uint32_t i = 0, *buf32;
+ uint32_t *buf32;
+ int i;
- /* verify that the len is a multiple of 4. see comment in
- * read_data_from_flash_mem() */
+ /*
+ * verify that the len is a multiple of 4.
+ * see comment in read_data_from_flash_mem()
+ */
BUG_ON((len % 4) != 0);
/* write the data to the flash memory */
buf32 = (uint32_t *)buf;
for (i = 0; i < len / 4; i++)
iowrite32(*buf32++, denali->flash_mem + 0x10);
- return i*4; /* intent is to return the number of bytes read */
+ return i * 4; /* intent is to return the number of bytes read */
}
/* helper function that simply reads a buffer from the flash */
static int read_data_from_flash_mem(struct denali_nand_info *denali,
- uint8_t *buf,
- int len)
+ uint8_t *buf, int len)
{
- uint32_t i = 0, *buf32;
-
- /* we assume that len will be a multiple of 4, if not
- * it would be nice to know about it ASAP rather than
- * have random failures...
- * This assumption is based on the fact that this
- * function is designed to be used to read flash pages,
- * which are typically multiples of 4...
- */
+ uint32_t *buf32;
+ int i;
+ /*
+ * we assume that len will be a multiple of 4, if not it would be nice
+ * to know about it ASAP rather than have random failures...
+ * This assumption is based on the fact that this function is designed
+ * to be used to read flash pages, which are typically multiples of 4.
+ */
BUG_ON((len % 4) != 0);
/* transfer the data from the flash */
buf32 = (uint32_t *)buf;
for (i = 0; i < len / 4; i++)
*buf32++ = ioread32(denali->flash_mem + 0x10);
- return i*4; /* intent is to return the number of bytes read */
+ return i * 4; /* intent is to return the number of bytes read */
}
/* writes OOB data to the device */
static int write_oob_data(struct mtd_info *mtd, uint8_t *buf, int page)
{
struct denali_nand_info *denali = mtd_to_denali(mtd);
- uint32_t irq_status = 0;
+ uint32_t irq_status;
uint32_t irq_mask = INTR_STATUS__PROGRAM_COMP |
INTR_STATUS__PROGRAM_FAIL;
int status = 0;
static void read_oob_data(struct mtd_info *mtd, uint8_t *buf, int page)
{
struct denali_nand_info *denali = mtd_to_denali(mtd);
- uint32_t irq_mask = INTR_STATUS__LOAD_COMP,
- irq_status = 0, addr = 0x0, cmd = 0x0;
+ uint32_t irq_mask = INTR_STATUS__LOAD_COMP;
+ uint32_t irq_status, addr, cmd;
denali->page = page;
DENALI_READ) == PASS) {
read_data_from_flash_mem(denali, buf, mtd->oobsize);
- /* wait for command to be accepted
- * can always use status0 bit as the mask is identical for each
- * bank. */
+ /*
+ * wait for command to be accepted
+ * can always use status0 bit as the
+ * mask is identical for each bank.
+ */
irq_status = wait_for_irq(denali, irq_mask);
if (irq_status == 0)
dev_err(denali->dev, "page on OOB timeout %d\n",
denali->page);
- /* We set the device back to MAIN_ACCESS here as I observed
+ /*
+ * We set the device back to MAIN_ACCESS here as I observed
* instability with the controller if you do a block erase
* and the last transaction was a SPARE_ACCESS. Block erase
* is reliable (according to the MTD test infrastructure)
*/
addr = BANK(denali->flash_bank) | denali->page;
cmd = MODE_10 | addr;
- index_addr(denali, (uint32_t)cmd, MAIN_ACCESS);
+ index_addr(denali, cmd, MAIN_ACCESS);
}
}
-/* this function examines buffers to see if they contain data that
+/*
+ * this function examines buffers to see if they contain data that
* indicate that the buffer is part of an erased region of flash.
*/
static bool is_erased(uint8_t *buf, int len)
{
- int i = 0;
+ int i;
+
for (i = 0; i < len; i++)
if (buf[i] != 0xFF)
return false;
if (irq_status & INTR_STATUS__ECC_ERR) {
/* read the ECC errors. we'll ignore them for now */
- uint32_t err_address = 0, err_correction_info = 0;
- uint32_t err_byte = 0, err_sector = 0, err_device = 0;
- uint32_t err_correction_value = 0;
+ uint32_t err_address, err_correction_info, err_byte,
+ err_sector, err_device, err_correction_value;
denali_set_intr_modes(denali, false);
do {
err_device = ECC_ERR_DEVICE(err_correction_info);
if (ECC_ERROR_CORRECTABLE(err_correction_info)) {
- /* If err_byte is larger than ECC_SECTOR_SIZE,
+ /*
+ * If err_byte is larger than ECC_SECTOR_SIZE,
* means error happened in OOB, so we ignore
* it. It's no need for us to correct it
* err_device is represented the NAND error
* bits are happened in if there are more
* than one NAND connected.
- * */
+ */
if (err_byte < ECC_SECTOR_SIZE) {
int offset;
+
offset = (err_sector *
ECC_SECTOR_SIZE +
err_byte) *
bitflips++;
}
} else {
- /* if the error is not correctable, need to
+ /*
+ * if the error is not correctable, need to
* look at the page to see if it is an erased
* page. if so, then it's not a real ECC error
- * */
+ */
check_erased_page = true;
}
} while (!ECC_LAST_ERR(err_correction_info));
- /* Once handle all ecc errors, controller will triger
+ /*
+ * Once handle all ecc errors, controller will triger
* a ECC_TRANSACTION_DONE interrupt, so here just wait
* for a while for this interrupt
- * */
+ */
while (!(read_interrupt_status(denali) &
INTR_STATUS__ECC_TRANSACTION_DONE))
cpu_relax();
/* programs the controller to either enable/disable DMA transfers */
static void denali_enable_dma(struct denali_nand_info *denali, bool en)
{
- uint32_t reg_val = 0x0;
-
- if (en)
- reg_val = DMA_ENABLE__FLAG;
-
- iowrite32(reg_val, denali->flash_reg + DMA_ENABLE);
+ iowrite32(en ? DMA_ENABLE__FLAG : 0, denali->flash_reg + DMA_ENABLE);
ioread32(denali->flash_reg + DMA_ENABLE);
}
/* setups the HW to perform the data DMA */
static void denali_setup_dma(struct denali_nand_info *denali, int op)
{
- uint32_t mode = 0x0;
+ uint32_t mode;
const int page_count = 1;
- dma_addr_t addr = denali->buf.dma_buf;
+ uint32_t addr = denali->buf.dma_buf;
mode = MODE_10 | BANK(denali->flash_bank);
index_addr(denali, mode | denali->page, 0x2000 | op | page_count);
/* 2. set memory high address bits 23:8 */
- index_addr(denali, mode | ((uint16_t)(addr >> 16) << 8), 0x2200);
+ index_addr(denali, mode | ((addr >> 16) << 8), 0x2200);
/* 3. set memory low address bits 23:8 */
- index_addr(denali, mode | ((uint16_t)addr << 8), 0x2300);
+ index_addr(denali, mode | ((addr & 0xff) << 8), 0x2300);
- /* 4. interrupt when complete, burst len = 64 bytes*/
+ /* 4. interrupt when complete, burst len = 64 bytes */
index_addr(denali, mode | 0x14000, 0x2400);
}
-/* writes a page. user specifies type, and this function handles the
- * configuration details. */
+/*
+ * writes a page. user specifies type, and this function handles the
+ * configuration details.
+ */
static int write_page(struct mtd_info *mtd, struct nand_chip *chip,
const uint8_t *buf, bool raw_xfer)
{
struct denali_nand_info *denali = mtd_to_denali(mtd);
-
dma_addr_t addr = denali->buf.dma_buf;
size_t size = denali->mtd.writesize + denali->mtd.oobsize;
-
- uint32_t irq_status = 0;
+ uint32_t irq_status;
uint32_t irq_mask = INTR_STATUS__DMA_CMD_COMP |
INTR_STATUS__PROGRAM_FAIL;
- /* if it is a raw xfer, we want to disable ecc, and send
- * the spare area.
+ /*
+ * if it is a raw xfer, we want to disable ecc and send the spare area.
* !raw_xfer - enable ecc
* raw_xfer - transfer spare
*/
irq_status = wait_for_irq(denali, irq_mask);
if (irq_status == 0) {
- dev_err(denali->dev,
- "timeout on write_page (type = %d)\n",
- raw_xfer);
- denali->status =
- (irq_status & INTR_STATUS__PROGRAM_FAIL) ?
- NAND_STATUS_FAIL : PASS;
+ dev_err(denali->dev, "timeout on write_page (type = %d)\n",
+ raw_xfer);
+ denali->status = NAND_STATUS_FAIL;
}
denali_enable_dma(denali, false);
/* NAND core entry points */
-/* this is the callback that the NAND core calls to write a page. Since
+/*
+ * this is the callback that the NAND core calls to write a page. Since
* writing a page with ECC or without is similar, all the work is done
* by write_page above.
- * */
+ */
static int denali_write_page(struct mtd_info *mtd, struct nand_chip *chip,
const uint8_t *buf, int oob_required)
{
- /* for regular page writes, we let HW handle all the ECC
- * data written to the device. */
+ /*
+ * for regular page writes, we let HW handle all the ECC
+ * data written to the device.
+ */
return write_page(mtd, chip, buf, false);
}
-/* This is the callback that the NAND core calls to write a page without ECC.
+/*
+ * This is the callback that the NAND core calls to write a page without ECC.
* raw access is similar to ECC page writes, so all the work is done in the
* write_page() function above.
*/
static int denali_write_page_raw(struct mtd_info *mtd, struct nand_chip *chip,
const uint8_t *buf, int oob_required)
{
- /* for raw page writes, we want to disable ECC and simply write
- whatever data is in the buffer. */
+ /*
+ * for raw page writes, we want to disable ECC and simply write
+ * whatever data is in the buffer.
+ */
return write_page(mtd, chip, buf, true);
}
dma_addr_t addr = denali->buf.dma_buf;
size_t size = denali->mtd.writesize + denali->mtd.oobsize;
- uint32_t irq_status = 0;
+ uint32_t irq_status;
uint32_t irq_mask = INTR_STATUS__ECC_TRANSACTION_DONE |
INTR_STATUS__ECC_ERR;
bool check_erased_page = false;
if (page != denali->page) {
- dev_err(denali->dev, "IN %s: page %d is not"
- " equal to denali->page %d, investigate!!",
- __func__, page, denali->page);
+ dev_err(denali->dev,
+ "IN %s: page %d is not equal to denali->page %d",
+ __func__, page, denali->page);
BUG();
}
uint8_t *buf, int oob_required, int page)
{
struct denali_nand_info *denali = mtd_to_denali(mtd);
-
dma_addr_t addr = denali->buf.dma_buf;
size_t size = denali->mtd.writesize + denali->mtd.oobsize;
-
- uint32_t irq_status = 0;
uint32_t irq_mask = INTR_STATUS__DMA_CMD_COMP;
if (page != denali->page) {
- dev_err(denali->dev, "IN %s: page %d is not"
- " equal to denali->page %d, investigate!!",
- __func__, page, denali->page);
+ dev_err(denali->dev,
+ "IN %s: page %d is not equal to denali->page %d",
+ __func__, page, denali->page);
BUG();
}
denali_setup_dma(denali, DENALI_READ);
/* wait for operation to complete */
- irq_status = wait_for_irq(denali, irq_mask);
+ wait_for_irq(denali, irq_mask);
dma_sync_single_for_cpu(denali->dev, addr, size, DMA_FROM_DEVICE);
{
struct denali_nand_info *denali = mtd_to_denali(mtd);
int status = denali->status;
+
denali->status = 0;
return status;
{
struct denali_nand_info *denali = mtd_to_denali(mtd);
- uint32_t cmd = 0x0, irq_status = 0;
+ uint32_t cmd, irq_status;
- /* clear interrupts */
clear_interrupts(denali);
/* setup page read request for access type */
cmd = MODE_10 | BANK(denali->flash_bank) | page;
- index_addr(denali, (uint32_t)cmd, 0x1);
+ index_addr(denali, cmd, 0x1);
/* wait for erase to complete or failure to occur */
irq_status = wait_for_irq(denali, INTR_STATUS__ERASE_COMP |
INTR_STATUS__ERASE_FAIL);
- return (irq_status & INTR_STATUS__ERASE_FAIL) ? NAND_STATUS_FAIL : PASS;
+ return irq_status & INTR_STATUS__ERASE_FAIL ? NAND_STATUS_FAIL : PASS;
}
static void denali_cmdfunc(struct mtd_info *mtd, unsigned int cmd, int col,
case NAND_CMD_READID:
case NAND_CMD_PARAM:
reset_buf(denali);
- /*sometimes ManufactureId read from register is not right
+ /*
+ * sometimes ManufactureId read from register is not right
* e.g. some of Micron MT29F32G08QAA MLC NAND chips
* So here we send READID cmd to NAND insteand
- * */
- addr = (uint32_t)MODE_11 | BANK(denali->flash_bank);
- index_addr(denali, (uint32_t)addr | 0, 0x90);
- index_addr(denali, (uint32_t)addr | 1, 0);
+ */
+ addr = MODE_11 | BANK(denali->flash_bank);
+ index_addr(denali, addr | 0, 0x90);
+ index_addr(denali, addr | 1, 0);
for (i = 0; i < 8; i++) {
- index_addr_read_data(denali,
- (uint32_t)addr | 2,
- &id);
+ index_addr_read_data(denali, addr | 2, &id);
write_byte_to_buf(denali, id);
}
break;
uint8_t *ecc_code)
{
struct denali_nand_info *denali = mtd_to_denali(mtd);
- dev_err(denali->dev,
- "denali_ecc_calculate called unexpectedly\n");
+
+ dev_err(denali->dev, "denali_ecc_calculate called unexpectedly\n");
BUG();
return -EIO;
}
uint8_t *read_ecc, uint8_t *calc_ecc)
{
struct denali_nand_info *denali = mtd_to_denali(mtd);
- dev_err(denali->dev,
- "denali_ecc_correct called unexpectedly\n");
+
+ dev_err(denali->dev, "denali_ecc_correct called unexpectedly\n");
BUG();
return -EIO;
}
static void denali_ecc_hwctl(struct mtd_info *mtd, int mode)
{
struct denali_nand_info *denali = mtd_to_denali(mtd);
- dev_err(denali->dev,
- "denali_ecc_hwctl called unexpectedly\n");
+
+ dev_err(denali->dev, "denali_ecc_hwctl called unexpectedly\n");
BUG();
}
/* end NAND core entry points */
/* Initialization code to bring the device up to a known good state */
static void denali_hw_init(struct denali_nand_info *denali)
{
- /* tell driver how many bit controller will skip before
+ /*
+ * tell driver how many bit controller will skip before
* writing ECC code in OOB, this register may be already
* set by firmware. So we read this value out.
* if this value is 0, just let it be.
- * */
+ */
denali->bbtskipbytes = ioread32(denali->flash_reg +
SPARE_AREA_SKIP_BYTES);
detect_max_banks(denali);
denali_irq_init(denali);
}
-/* Althogh controller spec said SLC ECC is forceb to be 4bit,
+/*
+ * Althogh controller spec said SLC ECC is forceb to be 4bit,
* but denali controller in MRST only support 15bit and 8bit ECC
* correction
- * */
+ */
#define ECC_8BITS 14
static struct nand_ecclayout nand_8bit_oob = {
.eccbytes = 14,
denali->idx = 0;
/* setup interrupt handler */
- /* the completion object will be used to notify
- * the callee that the interrupt is done */
+ /*
+ * the completion object will be used to notify
+ * the callee that the interrupt is done
+ */
init_completion(&denali->complete);
- /* the spinlock will be used to synchronize the ISR
- * with any element that might be access shared
- * data (interrupt status) */
+ /*
+ * the spinlock will be used to synchronize the ISR with any
+ * element that might be access shared data (interrupt status)
+ */
spin_lock_init(&denali->irq_lock);
/* indicate that MTD has not selected a valid bank yet */
int ret;
if (denali->platform == INTEL_CE4100) {
- /* Due to a silicon limitation, we can only support
+ /*
+ * Due to a silicon limitation, we can only support
* ONFI timing mode 1 and below.
*/
if (onfi_timing_mode < -1 || onfi_timing_mode > 1) {
denali_hw_init(denali);
denali_drv_init(denali);
- /* denali_isr register is done after all the hardware
- * initilization is finished*/
+ /*
+ * denali_isr register is done after all the hardware
+ * initilization is finished
+ */
if (request_irq(denali->irq, denali_isr, IRQF_SHARED,
DENALI_NAND_NAME, denali)) {
pr_err("Spectra: Unable to allocate IRQ\n");
denali->nand.read_byte = denali_read_byte;
denali->nand.waitfunc = denali_waitfunc;
- /* scan for NAND devices attached to the controller
+ /*
+ * scan for NAND devices attached to the controller
* this is the first stage in a two step process to register
- * with the nand subsystem */
+ * with the nand subsystem
+ */
if (nand_scan_ident(&denali->mtd, denali->max_banks, NULL)) {
ret = -ENXIO;
goto failed_req_irq;
goto failed_req_irq;
}
- /* support for multi nand
- * MTD known nothing about multi nand,
- * so we should tell it the real pagesize
- * and anything necessery
+ /*
+ * support for multi nand
+ * MTD known nothing about multi nand, so we should tell it
+ * the real pagesize and anything necessery
*/
denali->devnum = ioread32(denali->flash_reg + DEVICES_CONNECTED);
denali->nand.chipsize <<= (denali->devnum - 1);
denali->mtd.size = denali->nand.numchips * denali->nand.chipsize;
denali->bbtskipbytes *= denali->devnum;
- /* second stage of the NAND scan
+ /*
+ * second stage of the NAND scan
* this stage requires information regarding ECC and
- * bad block management. */
+ * bad block management.
+ */
/* Bad block management */
denali->nand.bbt_td = &bbt_main_descr;
denali->nand.options |= NAND_SKIP_BBTSCAN;
denali->nand.ecc.mode = NAND_ECC_HW_SYNDROME;
- /* Denali Controller only support 15bit and 8bit ECC in MRST,
+ /*
+ * Denali Controller only support 15bit and 8bit ECC in MRST,
* so just let controller do 15bit ECC for MLC and 8bit ECC for
* SLC if possible.
* */
} else if (denali->mtd.oobsize < (denali->bbtskipbytes +
ECC_8BITS * (denali->mtd.writesize /
ECC_SECTOR_SIZE))) {
- pr_err("Your NAND chip OOB is not large enough to \
- contain 8bit ECC correction codes");
+ pr_err("Your NAND chip OOB is not large enough to contain 8bit ECC correction codes");
goto failed_req_irq;
} else {
denali->nand.ecc.strength = 8;
denali->mtd.oobsize - denali->nand.ecc.layout->eccbytes -
denali->bbtskipbytes;
- /* Let driver know the total blocks number and
- * how many blocks contained by each nand chip.
- * blksperchip will help driver to know how many
- * blocks is taken by FW.
- * */
- denali->totalblks = denali->mtd.size >>
- denali->nand.phys_erase_shift;
+ /*
+ * Let driver know the total blocks number and how many blocks
+ * contained by each nand chip. blksperchip will help driver to
+ * know how many blocks is taken by FW.
+ */
+ denali->totalblks = denali->mtd.size >> denali->nand.phys_erase_shift;
denali->blksperchip = denali->totalblks / denali->nand.numchips;
- /* These functions are required by the NAND core framework, otherwise,
+ /*
+ * These functions are required by the NAND core framework, otherwise,
* the NAND core will assert. However, we don't need them, so we'll stub
- * them out. */
+ * them out.
+ */
denali->nand.ecc.calculate = denali_ecc_calculate;
denali->nand.ecc.correct = denali_ecc_correct;
denali->nand.ecc.hwctl = denali_ecc_hwctl;
{
denali_irq_cleanup(denali->irq, denali);
dma_unmap_single(denali->dev, denali->buf.dma_buf,
- denali->mtd.writesize + denali->mtd.oobsize,
- DMA_BIDIRECTIONAL);
+ denali->mtd.writesize + denali->mtd.oobsize,
+ DMA_BIDIRECTIONAL);
}
EXPORT_SYMBOL(denali_remove);
*
*/
+#ifndef __DENALI_H__
+#define __DENALI_H__
+
#include <linux/mtd/nand.h>
#define DEVICE_RESET 0x0
#define ONFI_BLOOM_TIME 1
#define MODE5_WORKAROUND 0
-/* lld_nand.h */
-/*
- * NAND Flash Controller Device Driver
- * Copyright (c) 2009, Intel Corporation and its suppliers.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
- *
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
- *
- */
-
-#ifndef _LLD_NAND_
-#define _LLD_NAND_
#define MODE_00 0x00000000
#define MODE_01 0x04000000
extern int denali_init(struct denali_nand_info *denali);
extern void denali_remove(struct denali_nand_info *denali);
-#endif /*_LLD_NAND_*/
+#endif /* __DENALI_H__ */
chip->select_chip(mtd, chipnr);
+ /*
+ * Reset the chip.
+ * If we want to check the WP through READ STATUS and check the bit 7
+ * we must reset the chip
+ * some operation can also clear the bit 7 of status register
+ * eg. erase/program a locked block
+ */
+ chip->cmdfunc(mtd, NAND_CMD_RESET, -1, -1);
+
/* Check, if it is write protected */
if (nand_check_wp(mtd)) {
pr_debug("%s: device is write protected!\n",
chip->select_chip(mtd, chipnr);
+ /*
+ * Reset the chip.
+ * If we want to check the WP through READ STATUS and check the bit 7
+ * we must reset the chip
+ * some operation can also clear the bit 7 of status register
+ * eg. erase/program a locked block
+ */
+ chip->cmdfunc(mtd, NAND_CMD_RESET, -1, -1);
+
/* Check, if it is write protected */
if (nand_check_wp(mtd)) {
pr_debug("%s: device is write protected!\n",
blockmask = (1 << (chip->phys_erase_shift - chip->page_shift)) - 1;
/* Invalidate the page cache, when we write to the cached page */
- if (to <= (chip->pagebuf << chip->page_shift) &&
- (chip->pagebuf << chip->page_shift) < (to + ops->len))
+ if (to <= ((loff_t)chip->pagebuf << chip->page_shift) &&
+ ((loff_t)chip->pagebuf << chip->page_shift) < (to + ops->len))
chip->pagebuf = -1;
/* Don't allow multipage oob writes with offset */
chip->options |= type->options;
chip->ecc_strength_ds = NAND_ECC_STRENGTH(type);
chip->ecc_step_ds = NAND_ECC_STEP(type);
+ chip->onfi_timing_mode_default =
+ type->onfi_timing_mode_default;
*busw = type->options & NAND_BUSWIDTH_16;
case NAND_ECC_HW_OOB_FIRST:
/* Similar to NAND_ECC_HW, but a separate read_page handle */
if (!ecc->calculate || !ecc->correct || !ecc->hwctl) {
- pr_warn("No ECC functions supplied; "
- "hardware ECC not possible\n");
+ pr_warn("No ECC functions supplied; hardware ECC not possible\n");
BUG();
}
if (!ecc->read_page)
ecc->read_page == nand_read_page_hwecc ||
!ecc->write_page ||
ecc->write_page == nand_write_page_hwecc)) {
- pr_warn("No ECC functions supplied; "
- "hardware ECC not possible\n");
+ pr_warn("No ECC functions supplied; hardware ECC not possible\n");
BUG();
}
/* Use standard syndrome read/write page function? */
}
break;
}
- pr_warn("%d byte HW ECC not possible on "
- "%d byte page size, fallback to SW ECC\n",
- ecc->size, mtd->writesize);
+ pr_warn("%d byte HW ECC not possible on %d byte page size, fallback to SW ECC\n",
+ ecc->size, mtd->writesize);
ecc->mode = NAND_ECC_SOFT;
case NAND_ECC_SOFT:
break;
case NAND_ECC_NONE:
- pr_warn("NAND_ECC_NONE selected by board driver. "
- "This is not recommended!\n");
+ pr_warn("NAND_ECC_NONE selected by board driver. This is not recommended!\n");
ecc->read_page = nand_read_page_raw;
ecc->write_page = nand_write_page_raw;
ecc->read_oob = nand_read_oob_std;
res = mtd_read(mtd, from, len, &retlen, buf);
if (res < 0) {
if (mtd_is_eccerr(res)) {
- pr_info("nand_bbt: ECC error in BBT at "
- "0x%012llx\n", from & ~mtd->writesize);
+ pr_info("nand_bbt: ECC error in BBT at 0x%012llx\n",
+ from & ~mtd->writesize);
return res;
} else if (mtd_is_bitflip(res)) {
- pr_info("nand_bbt: corrected error in BBT at "
- "0x%012llx\n", from & ~mtd->writesize);
+ pr_info("nand_bbt: corrected error in BBT at 0x%012llx\n",
+ from & ~mtd->writesize);
ret = res;
} else {
pr_info("nand_bbt: error reading BBT\n");
if (td->pages[i] == -1)
pr_warn("Bad block table not found for chip %d\n", i);
else
- pr_info("Bad block table found at page %d, version "
- "0x%02X\n", td->pages[i], td->version[i]);
+ pr_info("Bad block table found at page %d, version 0x%02X\n",
+ td->pages[i], td->version[i]);
}
return 0;
}
res = mtd_read(mtd, to, len, &retlen, buf);
if (res < 0) {
if (retlen != len) {
- pr_info("nand_bbt: error reading block "
- "for writing the bad block table\n");
+ pr_info("nand_bbt: error reading block for writing the bad block table\n");
return res;
}
- pr_warn("nand_bbt: ECC error while reading "
- "block for writing bad block table\n");
+ pr_warn("nand_bbt: ECC error while reading block for writing bad block table\n");
}
/* Read oob data */
ops.ooblen = (len >> this->page_shift) * mtd->oobsize;
block = (int)(offs >> this->bbt_erase_shift);
res = bbt_get_entry(this, block);
- pr_debug("nand_isbad_bbt(): bbt info for offs 0x%08x: "
- "(block %d) 0x%02x\n",
- (unsigned int)offs, block, res);
+ pr_debug("nand_isbad_bbt(): bbt info for offs 0x%08x: (block %d) 0x%02x\n",
+ (unsigned int)offs, block, res);
switch (res) {
case BBT_BLOCK_GOOD:
{"SDTNRGAMA 64G 3.3V 8-bit",
{ .id = {0x45, 0xde, 0x94, 0x93, 0x76, 0x50} },
SZ_16K, SZ_8K, SZ_4M, 0, 6, 1280, NAND_ECC_INFO(40, SZ_1K) },
+ {"H27UCG8T2ATR-BC 64G 3.3V 8-bit",
+ { .id = {0xad, 0xde, 0x94, 0xda, 0x74, 0xc4} },
+ SZ_8K, SZ_8K, SZ_2M, 0, 6, 640, NAND_ECC_INFO(40, SZ_1K),
+ 4 },
LEGACY_ID_NAND("NAND 4MiB 5V 8-bit", 0x6B, 4, SZ_8K, SP_OPTIONS),
LEGACY_ID_NAND("NAND 4MiB 3,3V 8-bit", 0xE3, 4, SZ_8K, SP_OPTIONS),
.tRHZ_max = 200000,
.tRLOH_min = 0,
.tRP_min = 50000,
- .tRST_max = 250000000000,
+ .tRST_max = 250000000000ULL,
.tWB_max = 200000,
.tRR_min = 40000,
.tWC_min = 100000,
NS_ERR("invalid badblocks.\n");
return -EINVAL;
}
- offset = erase_block_no * ns->geom.secsz;
+ offset = (loff_t)erase_block_no * ns->geom.secsz;
if (mtd_block_markbad(mtd, offset)) {
NS_ERR("invalid badblocks.\n");
return -EINVAL;
struct ndfc_controller *ndfc;
const __be32 *reg;
u32 ccr;
- int err, len, cs;
+ u32 cs;
+ int err, len;
/* Read the reg property to get the chip select */
reg = of_get_property(ofdev->dev.of_node, "reg", &len);
#define BADBLOCK_MARKER_LENGTH 2
-#ifdef CONFIG_MTD_NAND_OMAP_BCH
static u_char bch16_vector[] = {0xf5, 0x24, 0x1c, 0xd0, 0x61, 0xb3, 0xf1, 0x55,
0x2e, 0x2c, 0x86, 0xa3, 0xed, 0x36, 0x1b, 0x78,
0x48, 0x76, 0xa9, 0x3b, 0x97, 0xd1, 0x7a, 0x93,
static u_char bch8_vector[] = {0xf3, 0xdb, 0x14, 0x16, 0x8b, 0xd2, 0xbe, 0xcc,
0xac, 0x6b, 0xff, 0x99, 0x7b};
static u_char bch4_vector[] = {0x00, 0x6b, 0x31, 0xdd, 0x41, 0xbc, 0x10};
-#endif
/* oob info generated runtime depending on ecc algorithm and layout selected */
static struct nand_ecclayout omap_oobinfo;
return 0;
}
-#ifdef CONFIG_MTD_NAND_OMAP_BCH
/**
* erased_sector_bitflips - count bit flips
* @data: data sector buffer
erased_ecc_vec = bch16_vector;
break;
default:
- pr_err("invalid driver configuration\n");
+ dev_err(&info->pdev->dev, "invalid driver configuration\n");
return -EINVAL;
}
err = 0;
for (i = 0; i < eccsteps; i++) {
if (err_vec[i].error_uncorrectable) {
- pr_err("nand: uncorrectable bit-flips found\n");
+ dev_err(&info->pdev->dev,
+ "uncorrectable bit-flips found\n");
err = -EBADMSG;
} else if (err_vec[i].error_reported) {
for (j = 0; j < err_vec[i].error_count; j++) {
1 << bit_pos;
}
} else {
- pr_err("invalid bit-flip @ %d:%d\n",
- byte_pos, bit_pos);
+ dev_err(&info->pdev->dev,
+ "invalid bit-flip @ %d:%d\n",
+ byte_pos, bit_pos);
err = -EBADMSG;
}
}
/**
* is_elm_present - checks for presence of ELM module by scanning DT nodes
* @omap_nand_info: NAND device structure containing platform data
- * @bch_type: 0x0=BCH4, 0x1=BCH8, 0x2=BCH16
*/
-static int is_elm_present(struct omap_nand_info *info,
- struct device_node *elm_node, enum bch_ecc bch_type)
+static bool is_elm_present(struct omap_nand_info *info,
+ struct device_node *elm_node)
{
struct platform_device *pdev;
- struct nand_ecc_ctrl *ecc = &info->nand.ecc;
- int err;
+
/* check whether elm-id is passed via DT */
if (!elm_node) {
- pr_err("nand: error: ELM DT node not found\n");
- return -ENODEV;
+ dev_err(&info->pdev->dev, "ELM devicetree node not found\n");
+ return false;
}
pdev = of_find_device_by_node(elm_node);
/* check whether ELM device is registered */
if (!pdev) {
- pr_err("nand: error: ELM device not found\n");
- return -ENODEV;
+ dev_err(&info->pdev->dev, "ELM device not found\n");
+ return false;
}
/* ELM module available, now configure it */
info->elm_dev = &pdev->dev;
- err = elm_config(info->elm_dev, bch_type,
- (info->mtd.writesize / ecc->size), ecc->size, ecc->bytes);
+ return true;
+}
- return err;
+static bool omap2_nand_ecc_check(struct omap_nand_info *info,
+ struct omap_nand_platform_data *pdata)
+{
+ bool ecc_needs_bch, ecc_needs_omap_bch, ecc_needs_elm;
+
+ switch (info->ecc_opt) {
+ case OMAP_ECC_BCH4_CODE_HW_DETECTION_SW:
+ case OMAP_ECC_BCH8_CODE_HW_DETECTION_SW:
+ ecc_needs_omap_bch = false;
+ ecc_needs_bch = true;
+ ecc_needs_elm = false;
+ break;
+ case OMAP_ECC_BCH4_CODE_HW:
+ case OMAP_ECC_BCH8_CODE_HW:
+ case OMAP_ECC_BCH16_CODE_HW:
+ ecc_needs_omap_bch = true;
+ ecc_needs_bch = false;
+ ecc_needs_elm = true;
+ break;
+ default:
+ ecc_needs_omap_bch = false;
+ ecc_needs_bch = false;
+ ecc_needs_elm = false;
+ break;
+ }
+
+ if (ecc_needs_bch && !IS_ENABLED(CONFIG_MTD_NAND_ECC_BCH)) {
+ dev_err(&info->pdev->dev,
+ "CONFIG_MTD_NAND_ECC_BCH not enabled\n");
+ return false;
+ }
+ if (ecc_needs_omap_bch && !IS_ENABLED(CONFIG_MTD_NAND_OMAP_BCH)) {
+ dev_err(&info->pdev->dev,
+ "CONFIG_MTD_NAND_OMAP_BCH not enabled\n");
+ return false;
+ }
+ if (ecc_needs_elm && !is_elm_present(info, pdata->elm_of_node)) {
+ dev_err(&info->pdev->dev, "ELM not available\n");
+ return false;
+ }
+
+ return true;
}
-#endif /* CONFIG_MTD_NAND_ECC_BCH */
static int omap_nand_probe(struct platform_device *pdev)
{
mtd->owner = THIS_MODULE;
nand_chip = &info->nand;
nand_chip->ecc.priv = NULL;
- nand_chip->options |= NAND_SKIP_BBTSCAN;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
nand_chip->IO_ADDR_R = devm_ioremap_resource(&pdev->dev, res);
nand_chip->chip_delay = 50;
}
+ if (pdata->flash_bbt)
+ nand_chip->bbt_options |= NAND_BBT_USE_FLASH | NAND_BBT_NO_OOB;
+ else
+ nand_chip->options |= NAND_SKIP_BBTSCAN;
+
/* scan NAND device connected to chip controller */
nand_chip->options |= pdata->devsize & NAND_BUSWIDTH_16;
if (nand_scan_ident(mtd, 1, NULL)) {
- pr_err("nand device scan failed, may be bus-width mismatch\n");
+ dev_err(&info->pdev->dev, "scan failed, may be bus-width mismatch\n");
err = -ENXIO;
goto return_error;
}
/* check for small page devices */
if ((mtd->oobsize < 64) && (pdata->ecc_opt != OMAP_ECC_HAM1_CODE_HW)) {
- pr_err("small page devices are not supported\n");
+ dev_err(&info->pdev->dev, "small page devices are not supported\n");
err = -EINVAL;
goto return_error;
}
goto return_error;
}
+ if (!omap2_nand_ecc_check(info, pdata)) {
+ err = -EINVAL;
+ goto return_error;
+ }
+
/* populate MTD interface based on ECC scheme */
ecclayout = &omap_oobinfo;
switch (info->ecc_opt) {
break;
case OMAP_ECC_BCH4_CODE_HW_DETECTION_SW:
-#ifdef CONFIG_MTD_NAND_ECC_BCH
pr_info("nand: using OMAP_ECC_BCH4_CODE_HW_DETECTION_SW\n");
nand_chip->ecc.mode = NAND_ECC_HW;
nand_chip->ecc.size = 512;
nand_chip->ecc.bytes,
&ecclayout);
if (!nand_chip->ecc.priv) {
- pr_err("nand: error: unable to use s/w BCH library\n");
+ dev_err(&info->pdev->dev, "unable to use BCH library\n");
err = -EINVAL;
+ goto return_error;
}
break;
-#else
- pr_err("nand: error: CONFIG_MTD_NAND_ECC_BCH not enabled\n");
- err = -EINVAL;
- goto return_error;
-#endif
case OMAP_ECC_BCH4_CODE_HW:
-#ifdef CONFIG_MTD_NAND_OMAP_BCH
pr_info("nand: using OMAP_ECC_BCH4_CODE_HW ECC scheme\n");
nand_chip->ecc.mode = NAND_ECC_HW;
nand_chip->ecc.size = 512;
/* reserved marker already included in ecclayout->eccbytes */
ecclayout->oobfree->offset =
ecclayout->eccpos[ecclayout->eccbytes - 1] + 1;
- /* This ECC scheme requires ELM H/W block */
- if (is_elm_present(info, pdata->elm_of_node, BCH4_ECC) < 0) {
- pr_err("nand: error: could not initialize ELM\n");
- err = -ENODEV;
+
+ err = elm_config(info->elm_dev, BCH4_ECC,
+ info->mtd.writesize / nand_chip->ecc.size,
+ nand_chip->ecc.size, nand_chip->ecc.bytes);
+ if (err < 0)
goto return_error;
- }
break;
-#else
- pr_err("nand: error: CONFIG_MTD_NAND_OMAP_BCH not enabled\n");
- err = -EINVAL;
- goto return_error;
-#endif
case OMAP_ECC_BCH8_CODE_HW_DETECTION_SW:
-#ifdef CONFIG_MTD_NAND_ECC_BCH
pr_info("nand: using OMAP_ECC_BCH8_CODE_HW_DETECTION_SW\n");
nand_chip->ecc.mode = NAND_ECC_HW;
nand_chip->ecc.size = 512;
nand_chip->ecc.bytes,
&ecclayout);
if (!nand_chip->ecc.priv) {
- pr_err("nand: error: unable to use s/w BCH library\n");
+ dev_err(&info->pdev->dev, "unable to use BCH library\n");
err = -EINVAL;
goto return_error;
}
break;
-#else
- pr_err("nand: error: CONFIG_MTD_NAND_ECC_BCH not enabled\n");
- err = -EINVAL;
- goto return_error;
-#endif
case OMAP_ECC_BCH8_CODE_HW:
-#ifdef CONFIG_MTD_NAND_OMAP_BCH
pr_info("nand: using OMAP_ECC_BCH8_CODE_HW ECC scheme\n");
nand_chip->ecc.mode = NAND_ECC_HW;
nand_chip->ecc.size = 512;
nand_chip->ecc.calculate = omap_calculate_ecc_bch;
nand_chip->ecc.read_page = omap_read_page_bch;
nand_chip->ecc.write_page = omap_write_page_bch;
- /* This ECC scheme requires ELM H/W block */
- err = is_elm_present(info, pdata->elm_of_node, BCH8_ECC);
- if (err < 0) {
- pr_err("nand: error: could not initialize ELM\n");
+
+ err = elm_config(info->elm_dev, BCH8_ECC,
+ info->mtd.writesize / nand_chip->ecc.size,
+ nand_chip->ecc.size, nand_chip->ecc.bytes);
+ if (err < 0)
goto return_error;
- }
+
/* define ECC layout */
ecclayout->eccbytes = nand_chip->ecc.bytes *
(mtd->writesize /
ecclayout->oobfree->offset =
ecclayout->eccpos[ecclayout->eccbytes - 1] + 1;
break;
-#else
- pr_err("nand: error: CONFIG_MTD_NAND_OMAP_BCH not enabled\n");
- err = -EINVAL;
- goto return_error;
-#endif
case OMAP_ECC_BCH16_CODE_HW:
-#ifdef CONFIG_MTD_NAND_OMAP_BCH
pr_info("using OMAP_ECC_BCH16_CODE_HW ECC scheme\n");
nand_chip->ecc.mode = NAND_ECC_HW;
nand_chip->ecc.size = 512;
nand_chip->ecc.calculate = omap_calculate_ecc_bch;
nand_chip->ecc.read_page = omap_read_page_bch;
nand_chip->ecc.write_page = omap_write_page_bch;
- /* This ECC scheme requires ELM H/W block */
- err = is_elm_present(info, pdata->elm_of_node, BCH16_ECC);
- if (err < 0) {
- pr_err("ELM is required for this ECC scheme\n");
+
+ err = elm_config(info->elm_dev, BCH16_ECC,
+ info->mtd.writesize / nand_chip->ecc.size,
+ nand_chip->ecc.size, nand_chip->ecc.bytes);
+ if (err < 0)
goto return_error;
- }
+
/* define ECC layout */
ecclayout->eccbytes = nand_chip->ecc.bytes *
(mtd->writesize /
ecclayout->oobfree->offset =
ecclayout->eccpos[ecclayout->eccbytes - 1] + 1;
break;
-#else
- pr_err("nand: error: CONFIG_MTD_NAND_OMAP_BCH not enabled\n");
- err = -EINVAL;
- goto return_error;
-#endif
default:
- pr_err("nand: error: invalid or unsupported ECC scheme\n");
+ dev_err(&info->pdev->dev, "invalid or unsupported ECC scheme\n");
err = -EINVAL;
goto return_error;
}
ecclayout->oobfree->length = mtd->oobsize - ecclayout->oobfree->offset;
/* check if NAND device's OOB is enough to store ECC signatures */
if (mtd->oobsize < (ecclayout->eccbytes + BADBLOCK_MARKER_LENGTH)) {
- pr_err("not enough OOB bytes required = %d, available=%d\n",
- ecclayout->eccbytes, mtd->oobsize);
+ dev_err(&info->pdev->dev,
+ "not enough OOB bytes required = %d, available=%d\n",
+ ecclayout->eccbytes, mtd->oobsize);
err = -EINVAL;
goto return_error;
}
uint8_t ecc2[3];
uint8_t lba_copy2[2];
uint8_t ecc1[3];
-} __attribute__((packed));
+} __packed;
/* one sector is always 512 bytes, but it can consist of two nand pages */
{
struct sm_ftl *ftl = dev->priv;
struct ftl_zone *zone;
- int error, zone_num, block, boffset;
+ int error = 0, zone_num, block, boffset;
BUG_ON(ftl->readonly);
sm_break_offset(ftl, sec_no << 9, &zone_num, &block, &boffset);
if MTD_SPI_NOR
+config MTD_SPI_NOR_USE_4K_SECTORS
+ bool "Use small 4096 B erase sectors"
+ default y
+ help
+ Many flash memories support erasing small (4096 B) sectors. Depending
+ on the usage this feature may provide performance gain in comparison
+ to erasing whole blocks (32/64 KiB).
+ Changing a small part of the flash's contents is usually faster with
+ small sectors. On the other hand erasing should be faster when using
+ 64 KiB block instead of 16 × 4 KiB sectors.
+
+ Please note that some tools/drivers/filesystems may not work with
+ 4096 B erase size (e.g. UBIFS requires 15 KiB as a minimum).
+
config SPI_FSL_QUADSPI
tristate "Freescale Quad SPI controller"
depends on ARCH_MXC
{ "m25px32-s0", INFO(0x207316, 0, 64 * 1024, 64, SECT_4K) },
{ "m25px32-s1", INFO(0x206316, 0, 64 * 1024, 64, SECT_4K) },
{ "m25px64", INFO(0x207117, 0, 64 * 1024, 128, 0) },
+ { "m25px80", INFO(0x207114, 0, 64 * 1024, 16, 0) },
/* Winbond -- w25x "blocks" are 64K, "sectors" are 4KiB */
{ "w25x10", INFO(0xef3011, 0, 64 * 1024, 2, SECT_4K) },
{ "w25q32dw", INFO(0xef6016, 0, 64 * 1024, 64, SECT_4K) },
{ "w25x64", INFO(0xef3017, 0, 64 * 1024, 128, SECT_4K) },
{ "w25q64", INFO(0xef4017, 0, 64 * 1024, 128, SECT_4K) },
- { "w25q128", INFO(0xef4018, 0, 64 * 1024, 256, SECT_4K) },
{ "w25q80", INFO(0xef5014, 0, 64 * 1024, 16, SECT_4K) },
{ "w25q80bl", INFO(0xef4014, 0, 64 * 1024, 16, SECT_4K) },
{ "w25q128", INFO(0xef4018, 0, 64 * 1024, 256, SECT_4K) },
return ERR_PTR(-ENODEV);
}
-static const struct spi_device_id *jedec_probe(struct spi_nor *nor)
-{
- return nor->read_id(nor);
-}
-
static int spi_nor_read(struct mtd_info *mtd, loff_t from, size_t len,
size_t *retlen, u_char *buf)
{
enum read_mode mode)
{
struct flash_info *info;
- struct flash_platform_data *data;
struct device *dev = nor->dev;
struct mtd_info *mtd = nor->mtd;
struct device_node *np = dev->of_node;
if (ret)
return ret;
- /* Platform data helps sort out which chip type we have, as
- * well as how this board partitions it. If we don't have
- * a chip ID, try the JEDEC id commands; they'll work for most
- * newer chips, even if we don't recognize the particular chip.
- */
- data = dev_get_platdata(dev);
- if (data && data->type) {
- const struct spi_device_id *plat_id;
-
- for (i = 0; i < ARRAY_SIZE(spi_nor_ids) - 1; i++) {
- plat_id = &spi_nor_ids[i];
- if (strcmp(data->type, plat_id->name))
- continue;
- break;
- }
-
- if (i < ARRAY_SIZE(spi_nor_ids) - 1)
- id = plat_id;
- else
- dev_warn(dev, "unrecognized id %s\n", data->type);
- }
-
info = (void *)id->driver_data;
if (info->jedec_id) {
const struct spi_device_id *jid;
- jid = jedec_probe(nor);
+ jid = nor->read_id(nor);
if (IS_ERR(jid)) {
return PTR_ERR(jid);
} else if (jid != id) {
write_sr(nor, 0);
}
- if (data && data->name)
- mtd->name = data->name;
- else
+ if (!mtd->name)
mtd->name = dev_name(dev);
-
mtd->type = MTD_NORFLASH;
mtd->writesize = 1;
mtd->flags = MTD_CAP_NORFLASH;
nor->wait_till_ready == spi_nor_wait_till_ready)
nor->wait_till_ready = spi_nor_wait_till_fsr_ready;
+#ifdef CONFIG_MTD_SPI_NOR_USE_4K_SECTORS
/* prefer "small sector" erase if possible */
if (info->flags & SECT_4K) {
nor->erase_opcode = SPINOR_OP_BE_4K;
} else if (info->flags & SECT_4K_PMC) {
nor->erase_opcode = SPINOR_OP_BE_4K_PMC;
mtd->erasesize = 4096;
- } else {
+ } else
+#endif
+ {
nor->erase_opcode = SPINOR_OP_SE;
mtd->erasesize = info->sector_size;
}
{
int err;
struct erase_info ei;
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
memset(&ei, 0, sizeof(struct erase_info));
ei.mtd = mtd;
static int is_block_bad(struct mtd_info *mtd, unsigned int ebnum)
{
int ret;
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
ret = mtd_block_isbad(mtd, addr);
if (ret)
pr_info("Device uses %d subpages of %d bytes\n", subcount, subsize);
- offset = page_offset * mtd->writesize;
+ offset = (loff_t)page_offset * mtd->writesize;
eraseblock = mtd_div_by_eb(offset, mtd);
pr_info("Using page=%u, offset=%llu, eraseblock=%u\n",
int i;
struct mtd_oob_ops ops;
int err = 0;
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
prandom_bytes_state(&rnd_state, writebuf, use_len_max * pgcnt);
for (i = 0; i < pgcnt; ++i, addr += mtd->writesize) {
{
struct mtd_oob_ops ops;
int err = 0;
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
size_t len = mtd->ecclayout->oobavail * pgcnt;
prandom_bytes_state(&rnd_state, writebuf, len);
size_t sz = mtd->ecclayout->oobavail;
if (bbt[i] || bbt[i + 1])
continue;
- addr = (i + 1) * mtd->erasesize - mtd->writesize;
+ addr = (loff_t)(i + 1) * mtd->erasesize - mtd->writesize;
prandom_bytes_state(&rnd_state, writebuf, sz * cnt);
for (pg = 0; pg < cnt; ++pg) {
ops.mode = MTD_OPS_AUTO_OOB;
continue;
prandom_bytes_state(&rnd_state, writebuf,
mtd->ecclayout->oobavail * 2);
- addr = (i + 1) * mtd->erasesize - mtd->writesize;
+ addr = (loff_t)(i + 1) * mtd->erasesize - mtd->writesize;
ops.mode = MTD_OPS_AUTO_OOB;
ops.len = 0;
ops.retlen = 0;
static int write_eraseblock(int ebnum)
{
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
prandom_bytes_state(&rnd_state, writebuf, mtd->erasesize);
cond_resched();
uint32_t j;
int err = 0, i;
loff_t addr0, addrn;
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
addr0 = 0;
for (i = 0; i < ebcnt && bbt[i]; ++i)
static int read_eraseblock_by_page(int ebnum)
{
int i, ret, err = 0;
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
void *buf = iobuf;
void *oobbuf = iobuf1;
{
int err;
struct erase_info ei;
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
memset(&ei, 0, sizeof(struct erase_info));
ei.mtd = mtd;
static int write_eraseblock(int ebnum)
{
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
return mtdtest_write(mtd, addr, mtd->erasesize, iobuf);
}
static int write_eraseblock_by_page(int ebnum)
{
int i, err = 0;
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
void *buf = iobuf;
for (i = 0; i < pgcnt; i++) {
{
size_t sz = pgsize * 2;
int i, n = pgcnt / 2, err = 0;
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
void *buf = iobuf;
for (i = 0; i < n; i++) {
static int read_eraseblock(int ebnum)
{
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
return mtdtest_read(mtd, addr, mtd->erasesize, iobuf);
}
static int read_eraseblock_by_page(int ebnum)
{
int i, err = 0;
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
void *buf = iobuf;
for (i = 0; i < pgcnt; i++) {
{
size_t sz = pgsize * 2;
int i, n = pgcnt / 2, err = 0;
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
void *buf = iobuf;
for (i = 0; i < n; i++) {
{
size_t written;
int err = 0;
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
prandom_bytes_state(&rnd_state, writebuf, subpgsize);
err = mtd_write(mtd, addr, subpgsize, &written, writebuf);
{
size_t written;
int err = 0, k;
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
for (k = 1; k < 33; ++k) {
if (addr + (subpgsize * k) > (ebnum + 1) * mtd->erasesize)
{
size_t read;
int err = 0;
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
prandom_bytes_state(&rnd_state, writebuf, subpgsize);
clear_data(readbuf, subpgsize);
{
size_t read;
int err = 0, k;
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
for (k = 1; k < 33; ++k) {
if (addr + (subpgsize * k) > (ebnum + 1) * mtd->erasesize)
uint32_t j;
size_t read;
int err = 0;
- loff_t addr = ebnum * mtd->erasesize;
+ loff_t addr = (loff_t)ebnum * mtd->erasesize;
memset(writebuf, 0xff, subpgsize);
for (j = 0; j < mtd->erasesize / subpgsize; ++j) {
static int reg_read(struct dsa_switch *ds, int addr, int reg)
{
- return mdiobus_read(to_mii_bus(ds->master_dev),
- ds->pd->sw_addr + addr, reg);
+ struct mii_bus *bus = dsa_host_dev_to_mii_bus(ds->master_dev);
+
+ if (bus == NULL)
+ return -EINVAL;
+
+ return mdiobus_read(bus, ds->pd->sw_addr + addr, reg);
}
#define REG_READ(addr, reg) \
static int reg_write(struct dsa_switch *ds, int addr, int reg, u16 val)
{
- return mdiobus_write(to_mii_bus(ds->master_dev),
- ds->pd->sw_addr + addr, reg, val);
+ struct mii_bus *bus = dsa_host_dev_to_mii_bus(ds->master_dev);
+
+ if (bus == NULL)
+ return -EINVAL;
+
+ return mdiobus_write(bus, ds->pd->sw_addr + addr, reg, val);
}
#define REG_WRITE(addr, reg, val) \
*/
val = 0x0433;
if (dsa_is_cpu_port(ds, p)) {
- if (ds->dst->tag_protocol == htons(ETH_P_EDSA))
+ if (ds->dst->tag_protocol == DSA_TAG_PROTO_EDSA)
val |= 0x3300;
else
val |= 0x0100;
int mv88e6xxx_reg_read(struct dsa_switch *ds, int addr, int reg)
{
struct mv88e6xxx_priv_state *ps = ds_to_priv(ds);
+ struct mii_bus *bus = dsa_host_dev_to_mii_bus(ds->master_dev);
int ret;
+ if (bus == NULL)
+ return -EINVAL;
+
mutex_lock(&ps->smi_mutex);
- ret = __mv88e6xxx_reg_read(to_mii_bus(ds->master_dev),
- ds->pd->sw_addr, addr, reg);
+ ret = __mv88e6xxx_reg_read(bus, ds->pd->sw_addr, addr, reg);
mutex_unlock(&ps->smi_mutex);
return ret;
int mv88e6xxx_reg_write(struct dsa_switch *ds, int addr, int reg, u16 val)
{
struct mv88e6xxx_priv_state *ps = ds_to_priv(ds);
+ struct mii_bus *bus = dsa_host_dev_to_mii_bus(ds->master_dev);
int ret;
+ if (bus == NULL)
+ return -EINVAL;
+
mutex_lock(&ps->smi_mutex);
- ret = __mv88e6xxx_reg_write(to_mii_bus(ds->master_dev),
- ds->pd->sw_addr, addr, reg, val);
+ ret = __mv88e6xxx_reg_write(bus, ds->pd->sw_addr, addr, reg, val);
mutex_unlock(&ps->smi_mutex);
return ret;
# Makefile for APM X-Gene Ethernet Driver.
#
-xgene-enet-objs := xgene_enet_hw.o xgene_enet_xgmac.o \
+xgene-enet-objs := xgene_enet_hw.o xgene_enet_sgmac.o xgene_enet_xgmac.o \
xgene_enet_main.o xgene_enet_ethtool.o
obj-$(CONFIG_NET_XGENE) += xgene-enet.o
return -ENODEV;
return phy_ethtool_gset(phydev, cmd);
+ } else if (pdata->phy_mode == PHY_INTERFACE_MODE_SGMII) {
+ cmd->supported = SUPPORTED_1000baseT_Full |
+ SUPPORTED_Autoneg | SUPPORTED_MII;
+ cmd->advertising = cmd->supported;
+ ethtool_cmd_speed_set(cmd, SPEED_1000);
+ cmd->duplex = DUPLEX_FULL;
+ cmd->port = PORT_MII;
+ cmd->transceiver = XCVR_INTERNAL;
+ cmd->autoneg = AUTONEG_ENABLE;
+ } else {
+ cmd->supported = SUPPORTED_10000baseT_Full | SUPPORTED_FIBRE;
+ cmd->advertising = cmd->supported;
+ ethtool_cmd_speed_set(cmd, SPEED_10000);
+ cmd->duplex = DUPLEX_FULL;
+ cmd->port = PORT_FIBRE;
+ cmd->transceiver = XCVR_INTERNAL;
+ cmd->autoneg = AUTONEG_DISABLE;
}
- cmd->supported = SUPPORTED_10000baseT_Full | SUPPORTED_FIBRE;
- cmd->advertising = cmd->supported;
- ethtool_cmd_speed_set(cmd, SPEED_10000);
- cmd->duplex = DUPLEX_FULL;
- cmd->port = PORT_FIBRE;
- cmd->transceiver = XCVR_EXTERNAL;
- cmd->autoneg = AUTONEG_DISABLE;
-
return 0;
}
addr0 = (dev_addr[3] << 24) | (dev_addr[2] << 16) |
(dev_addr[1] << 8) | dev_addr[0];
addr1 = (dev_addr[5] << 24) | (dev_addr[4] << 16);
- addr1 |= pdata->phy_addr & 0xFFFF;
xgene_enet_wr_mcx_mac(pdata, STATION_ADDR0_ADDR, addr0);
xgene_enet_wr_mcx_mac(pdata, STATION_ADDR1_ADDR, addr1);
enum xgene_enet_rm {
RM0,
+ RM1,
RM3 = 3
};
#define CFG_CLE_FPSEL0_SET(dst, val) xgene_set_bits(dst, val, 16, 4)
#define CFG_MACMODE_SET(dst, val) xgene_set_bits(dst, val, 18, 2)
#define CFG_WAITASYNCRD_SET(dst, val) xgene_set_bits(dst, val, 0, 16)
+#define CFG_CLE_DSTQID0(val) (val & GENMASK(11, 0))
+#define CFG_CLE_FPSEL0(val) ((val << 16) & GENMASK(19, 16))
#define ICM_CONFIG0_REG_0_ADDR 0x0400
#define ICM_CONFIG2_REG_0_ADDR 0x0410
#define RX_DV_GATE_REG_0_ADDR 0x05fc
#define TUND_ADDR 0x4a
#define TSO_IPPROTO_TCP 1
-#define FULL_DUPLEX 2
#define USERINFO_POS 0
#define USERINFO_LEN 32
#include "xgene_enet_main.h"
#include "xgene_enet_hw.h"
+#include "xgene_enet_sgmac.h"
#include "xgene_enet_xgmac.h"
static void xgene_enet_init_bufpool(struct xgene_enet_desc_ring *buf_pool)
return pdata->phy_mode;
}
if (pdata->phy_mode != PHY_INTERFACE_MODE_RGMII &&
+ pdata->phy_mode != PHY_INTERFACE_MODE_SGMII &&
pdata->phy_mode != PHY_INTERFACE_MODE_XGMII) {
dev_err(dev, "Incorrect phy-connection-type specified\n");
return -ENODEV;
pdata->eth_csr_addr = base_addr + BLOCK_ETH_CSR_OFFSET;
pdata->eth_ring_if_addr = base_addr + BLOCK_ETH_RING_IF_OFFSET;
pdata->eth_diag_csr_addr = base_addr + BLOCK_ETH_DIAG_CSR_OFFSET;
- if (pdata->phy_mode == PHY_INTERFACE_MODE_RGMII) {
+ if (pdata->phy_mode == PHY_INTERFACE_MODE_RGMII ||
+ pdata->phy_mode == PHY_INTERFACE_MODE_SGMII) {
pdata->mcx_mac_addr = base_addr + BLOCK_ETH_MAC_OFFSET;
pdata->mcx_mac_csr_addr = base_addr + BLOCK_ETH_MAC_CSR_OFFSET;
- pdata->rm = RM3;
} else {
pdata->mcx_mac_addr = base_addr + BLOCK_AXG_MAC_OFFSET;
pdata->mcx_mac_csr_addr = base_addr + BLOCK_AXG_MAC_CSR_OFFSET;
- pdata->rm = RM0;
}
pdata->rx_buff_cnt = NUM_PKT_BUF;
case PHY_INTERFACE_MODE_RGMII:
pdata->mac_ops = &xgene_gmac_ops;
pdata->port_ops = &xgene_gport_ops;
+ pdata->rm = RM3;
+ break;
+ case PHY_INTERFACE_MODE_SGMII:
+ pdata->mac_ops = &xgene_sgmac_ops;
+ pdata->port_ops = &xgene_sgport_ops;
+ pdata->rm = RM1;
break;
default:
pdata->mac_ops = &xgene_xgmac_ops;
pdata->port_ops = &xgene_xgport_ops;
+ pdata->rm = RM0;
break;
}
}
struct xgene_enet_pdata *pdata;
struct device *dev = &pdev->dev;
struct napi_struct *napi;
+ struct xgene_mac_ops *mac_ops;
int ret;
ndev = alloc_etherdev(sizeof(struct xgene_enet_pdata));
napi = &pdata->rx_ring->napi;
netif_napi_add(ndev, napi, xgene_enet_napi, NAPI_POLL_WEIGHT);
+ mac_ops = pdata->mac_ops;
if (pdata->phy_mode == PHY_INTERFACE_MODE_RGMII)
ret = xgene_enet_mdio_config(pdata);
else
- INIT_DELAYED_WORK(&pdata->link_work, xgene_enet_link_state);
+ INIT_DELAYED_WORK(&pdata->link_work, mac_ops->link_state);
return ret;
err:
#define NUM_PKT_BUF 64
#define NUM_BUFPOOL 32
+#define PHY_POLL_LINK_ON (10 * HZ)
+#define PHY_POLL_LINK_OFF (PHY_POLL_LINK_ON / 5)
+
/* software context of a descriptor ring */
struct xgene_enet_desc_ring {
struct net_device *ndev;
void (*tx_disable)(struct xgene_enet_pdata *pdata);
void (*rx_disable)(struct xgene_enet_pdata *pdata);
void (*set_mac_addr)(struct xgene_enet_pdata *pdata);
+ void (*link_state)(struct work_struct *work);
};
struct xgene_port_ops {
void __iomem *base_addr;
void __iomem *ring_csr_addr;
void __iomem *ring_cmd_addr;
- u32 phy_addr;
int phy_mode;
enum xgene_enet_rm rm;
struct rtnl_link_stats64 stats;
struct delayed_work link_work;
};
+struct xgene_indirect_ctl {
+ void __iomem *addr;
+ void __iomem *ctl;
+ void __iomem *cmd;
+ void __iomem *cmd_done;
+};
+
/* Set the specified value into a bit-field defined by its starting position
* and length within a single u64.
*/
--- /dev/null
+/* Applied Micro X-Gene SoC Ethernet Driver
+ *
+ * Copyright (c) 2014, Applied Micro Circuits Corporation
+ * Authors: Iyappan Subramanian <isubramanian@apm.com>
+ * Keyur Chudgar <kchudgar@apm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "xgene_enet_main.h"
+#include "xgene_enet_hw.h"
+#include "xgene_enet_sgmac.h"
+
+static void xgene_enet_wr_csr(struct xgene_enet_pdata *p, u32 offset, u32 val)
+{
+ iowrite32(val, p->eth_csr_addr + offset);
+}
+
+static void xgene_enet_wr_ring_if(struct xgene_enet_pdata *p,
+ u32 offset, u32 val)
+{
+ iowrite32(val, p->eth_ring_if_addr + offset);
+}
+
+static void xgene_enet_wr_diag_csr(struct xgene_enet_pdata *p,
+ u32 offset, u32 val)
+{
+ iowrite32(val, p->eth_diag_csr_addr + offset);
+}
+
+static bool xgene_enet_wr_indirect(struct xgene_indirect_ctl *ctl,
+ u32 wr_addr, u32 wr_data)
+{
+ int i;
+
+ iowrite32(wr_addr, ctl->addr);
+ iowrite32(wr_data, ctl->ctl);
+ iowrite32(XGENE_ENET_WR_CMD, ctl->cmd);
+
+ /* wait for write command to complete */
+ for (i = 0; i < 10; i++) {
+ if (ioread32(ctl->cmd_done)) {
+ iowrite32(0, ctl->cmd);
+ return true;
+ }
+ udelay(1);
+ }
+
+ return false;
+}
+
+static void xgene_enet_wr_mac(struct xgene_enet_pdata *p,
+ u32 wr_addr, u32 wr_data)
+{
+ struct xgene_indirect_ctl ctl = {
+ .addr = p->mcx_mac_addr + MAC_ADDR_REG_OFFSET,
+ .ctl = p->mcx_mac_addr + MAC_WRITE_REG_OFFSET,
+ .cmd = p->mcx_mac_addr + MAC_COMMAND_REG_OFFSET,
+ .cmd_done = p->mcx_mac_addr + MAC_COMMAND_DONE_REG_OFFSET
+ };
+
+ if (!xgene_enet_wr_indirect(&ctl, wr_addr, wr_data))
+ netdev_err(p->ndev, "mac write failed, addr: %04x\n", wr_addr);
+}
+
+static u32 xgene_enet_rd_csr(struct xgene_enet_pdata *p, u32 offset)
+{
+ return ioread32(p->eth_csr_addr + offset);
+}
+
+static u32 xgene_enet_rd_diag_csr(struct xgene_enet_pdata *p, u32 offset)
+{
+ return ioread32(p->eth_diag_csr_addr + offset);
+}
+
+static u32 xgene_enet_rd_indirect(struct xgene_indirect_ctl *ctl, u32 rd_addr)
+{
+ u32 rd_data;
+ int i;
+
+ iowrite32(rd_addr, ctl->addr);
+ iowrite32(XGENE_ENET_RD_CMD, ctl->cmd);
+
+ /* wait for read command to complete */
+ for (i = 0; i < 10; i++) {
+ if (ioread32(ctl->cmd_done)) {
+ rd_data = ioread32(ctl->ctl);
+ iowrite32(0, ctl->cmd);
+
+ return rd_data;
+ }
+ udelay(1);
+ }
+
+ pr_err("%s: mac read failed, addr: %04x\n", __func__, rd_addr);
+
+ return 0;
+}
+
+static u32 xgene_enet_rd_mac(struct xgene_enet_pdata *p, u32 rd_addr)
+{
+ struct xgene_indirect_ctl ctl = {
+ .addr = p->mcx_mac_addr + MAC_ADDR_REG_OFFSET,
+ .ctl = p->mcx_mac_addr + MAC_READ_REG_OFFSET,
+ .cmd = p->mcx_mac_addr + MAC_COMMAND_REG_OFFSET,
+ .cmd_done = p->mcx_mac_addr + MAC_COMMAND_DONE_REG_OFFSET
+ };
+
+ return xgene_enet_rd_indirect(&ctl, rd_addr);
+}
+
+static int xgene_enet_ecc_init(struct xgene_enet_pdata *p)
+{
+ struct net_device *ndev = p->ndev;
+ u32 data;
+ int i;
+
+ xgene_enet_wr_diag_csr(p, ENET_CFG_MEM_RAM_SHUTDOWN_ADDR, 0);
+ for (i = 0; i < 10 && data != ~0U ; i++) {
+ usleep_range(100, 110);
+ data = xgene_enet_rd_diag_csr(p, ENET_BLOCK_MEM_RDY_ADDR);
+ }
+
+ if (data != ~0U) {
+ netdev_err(ndev, "Failed to release memory from shutdown\n");
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
+static void xgene_enet_config_ring_if_assoc(struct xgene_enet_pdata *p)
+{
+ u32 val = 0xffffffff;
+
+ xgene_enet_wr_ring_if(p, ENET_CFGSSQMIWQASSOC_ADDR, val);
+ xgene_enet_wr_ring_if(p, ENET_CFGSSQMIFPQASSOC_ADDR, val);
+}
+
+static void xgene_mii_phy_write(struct xgene_enet_pdata *p, u8 phy_id,
+ u32 reg, u16 data)
+{
+ u32 addr, wr_data, done;
+ int i;
+
+ addr = PHY_ADDR(phy_id) | REG_ADDR(reg);
+ xgene_enet_wr_mac(p, MII_MGMT_ADDRESS_ADDR, addr);
+
+ wr_data = PHY_CONTROL(data);
+ xgene_enet_wr_mac(p, MII_MGMT_CONTROL_ADDR, wr_data);
+
+ for (i = 0; i < 10; i++) {
+ done = xgene_enet_rd_mac(p, MII_MGMT_INDICATORS_ADDR);
+ if (!(done & BUSY_MASK))
+ return;
+ usleep_range(10, 20);
+ }
+
+ netdev_err(p->ndev, "MII_MGMT write failed\n");
+}
+
+static u32 xgene_mii_phy_read(struct xgene_enet_pdata *p, u8 phy_id, u32 reg)
+{
+ u32 addr, data, done;
+ int i;
+
+ addr = PHY_ADDR(phy_id) | REG_ADDR(reg);
+ xgene_enet_wr_mac(p, MII_MGMT_ADDRESS_ADDR, addr);
+ xgene_enet_wr_mac(p, MII_MGMT_COMMAND_ADDR, READ_CYCLE_MASK);
+
+ for (i = 0; i < 10; i++) {
+ done = xgene_enet_rd_mac(p, MII_MGMT_INDICATORS_ADDR);
+ if (!(done & BUSY_MASK)) {
+ data = xgene_enet_rd_mac(p, MII_MGMT_STATUS_ADDR);
+ xgene_enet_wr_mac(p, MII_MGMT_COMMAND_ADDR, 0);
+
+ return data;
+ }
+ usleep_range(10, 20);
+ }
+
+ netdev_err(p->ndev, "MII_MGMT read failed\n");
+
+ return 0;
+}
+
+static void xgene_sgmac_reset(struct xgene_enet_pdata *p)
+{
+ xgene_enet_wr_mac(p, MAC_CONFIG_1_ADDR, SOFT_RESET1);
+ xgene_enet_wr_mac(p, MAC_CONFIG_1_ADDR, 0);
+}
+
+static void xgene_sgmac_set_mac_addr(struct xgene_enet_pdata *p)
+{
+ u32 addr0, addr1;
+ u8 *dev_addr = p->ndev->dev_addr;
+
+ addr0 = (dev_addr[3] << 24) | (dev_addr[2] << 16) |
+ (dev_addr[1] << 8) | dev_addr[0];
+ xgene_enet_wr_mac(p, STATION_ADDR0_ADDR, addr0);
+
+ addr1 = xgene_enet_rd_mac(p, STATION_ADDR1_ADDR);
+ addr1 |= (dev_addr[5] << 24) | (dev_addr[4] << 16);
+ xgene_enet_wr_mac(p, STATION_ADDR1_ADDR, addr1);
+}
+
+static u32 xgene_enet_link_status(struct xgene_enet_pdata *p)
+{
+ u32 data;
+
+ data = xgene_mii_phy_read(p, INT_PHY_ADDR,
+ SGMII_BASE_PAGE_ABILITY_ADDR >> 2);
+
+ return data & LINK_UP;
+}
+
+static void xgene_sgmac_init(struct xgene_enet_pdata *p)
+{
+ u32 data, loop = 10;
+
+ xgene_sgmac_reset(p);
+
+ /* Enable auto-negotiation */
+ xgene_mii_phy_write(p, INT_PHY_ADDR, SGMII_CONTROL_ADDR >> 2, 0x1000);
+ xgene_mii_phy_write(p, INT_PHY_ADDR, SGMII_TBI_CONTROL_ADDR >> 2, 0);
+
+ while (loop--) {
+ data = xgene_mii_phy_read(p, INT_PHY_ADDR,
+ SGMII_STATUS_ADDR >> 2);
+ if ((data & AUTO_NEG_COMPLETE) && (data & LINK_STATUS))
+ break;
+ usleep_range(10, 20);
+ }
+ if (!(data & AUTO_NEG_COMPLETE) || !(data & LINK_STATUS))
+ netdev_err(p->ndev, "Auto-negotiation failed\n");
+
+ data = xgene_enet_rd_mac(p, MAC_CONFIG_2_ADDR);
+ ENET_INTERFACE_MODE2_SET(&data, 2);
+ xgene_enet_wr_mac(p, MAC_CONFIG_2_ADDR, data | FULL_DUPLEX2);
+ xgene_enet_wr_mac(p, INTERFACE_CONTROL_ADDR, ENET_GHD_MODE);
+
+ data = xgene_enet_rd_csr(p, ENET_SPARE_CFG_REG_ADDR);
+ data |= MPA_IDLE_WITH_QMI_EMPTY;
+ xgene_enet_wr_csr(p, ENET_SPARE_CFG_REG_ADDR, data);
+
+ xgene_sgmac_set_mac_addr(p);
+
+ data = xgene_enet_rd_csr(p, DEBUG_REG_ADDR);
+ data |= CFG_BYPASS_UNISEC_TX | CFG_BYPASS_UNISEC_RX;
+ xgene_enet_wr_csr(p, DEBUG_REG_ADDR, data);
+
+ /* Adjust MDC clock frequency */
+ data = xgene_enet_rd_mac(p, MII_MGMT_CONFIG_ADDR);
+ MGMT_CLOCK_SEL_SET(&data, 7);
+ xgene_enet_wr_mac(p, MII_MGMT_CONFIG_ADDR, data);
+
+ /* Enable drop if bufpool not available */
+ data = xgene_enet_rd_csr(p, RSIF_CONFIG_REG_ADDR);
+ data |= CFG_RSIF_FPBUFF_TIMEOUT_EN;
+ xgene_enet_wr_csr(p, RSIF_CONFIG_REG_ADDR, data);
+
+ /* Rtype should be copied from FP */
+ xgene_enet_wr_csr(p, RSIF_RAM_DBG_REG0_ADDR, 0);
+
+ /* Bypass traffic gating */
+ xgene_enet_wr_csr(p, CFG_LINK_AGGR_RESUME_0_ADDR, TX_PORT0);
+ xgene_enet_wr_csr(p, CFG_BYPASS_ADDR, RESUME_TX);
+ xgene_enet_wr_csr(p, SG_RX_DV_GATE_REG_0_ADDR, RESUME_RX0);
+}
+
+static void xgene_sgmac_rxtx(struct xgene_enet_pdata *p, u32 bits, bool set)
+{
+ u32 data;
+
+ data = xgene_enet_rd_mac(p, MAC_CONFIG_1_ADDR);
+
+ if (set)
+ data |= bits;
+ else
+ data &= ~bits;
+
+ xgene_enet_wr_mac(p, MAC_CONFIG_1_ADDR, data);
+}
+
+static void xgene_sgmac_rx_enable(struct xgene_enet_pdata *p)
+{
+ xgene_sgmac_rxtx(p, RX_EN, true);
+}
+
+static void xgene_sgmac_tx_enable(struct xgene_enet_pdata *p)
+{
+ xgene_sgmac_rxtx(p, TX_EN, true);
+}
+
+static void xgene_sgmac_rx_disable(struct xgene_enet_pdata *p)
+{
+ xgene_sgmac_rxtx(p, RX_EN, false);
+}
+
+static void xgene_sgmac_tx_disable(struct xgene_enet_pdata *p)
+{
+ xgene_sgmac_rxtx(p, TX_EN, false);
+}
+
+static void xgene_enet_reset(struct xgene_enet_pdata *p)
+{
+ clk_prepare_enable(p->clk);
+ clk_disable_unprepare(p->clk);
+ clk_prepare_enable(p->clk);
+
+ xgene_enet_ecc_init(p);
+ xgene_enet_config_ring_if_assoc(p);
+}
+
+static void xgene_enet_cle_bypass(struct xgene_enet_pdata *p,
+ u32 dst_ring_num, u16 bufpool_id)
+{
+ u32 data, fpsel;
+
+ data = CFG_CLE_BYPASS_EN0;
+ xgene_enet_wr_csr(p, CLE_BYPASS_REG0_0_ADDR, data);
+
+ fpsel = xgene_enet_ring_bufnum(bufpool_id) - 0x20;
+ data = CFG_CLE_DSTQID0(dst_ring_num) | CFG_CLE_FPSEL0(fpsel);
+ xgene_enet_wr_csr(p, CLE_BYPASS_REG1_0_ADDR, data);
+}
+
+static void xgene_enet_shutdown(struct xgene_enet_pdata *p)
+{
+ clk_disable_unprepare(p->clk);
+}
+
+static void xgene_enet_link_state(struct work_struct *work)
+{
+ struct xgene_enet_pdata *p = container_of(to_delayed_work(work),
+ struct xgene_enet_pdata, link_work);
+ struct net_device *ndev = p->ndev;
+ u32 link, poll_interval;
+
+ link = xgene_enet_link_status(p);
+ if (link) {
+ if (!netif_carrier_ok(ndev)) {
+ netif_carrier_on(ndev);
+ xgene_sgmac_init(p);
+ xgene_sgmac_rx_enable(p);
+ xgene_sgmac_tx_enable(p);
+ netdev_info(ndev, "Link is Up - 1Gbps\n");
+ }
+ poll_interval = PHY_POLL_LINK_ON;
+ } else {
+ if (netif_carrier_ok(ndev)) {
+ xgene_sgmac_rx_disable(p);
+ xgene_sgmac_tx_disable(p);
+ netif_carrier_off(ndev);
+ netdev_info(ndev, "Link is Down\n");
+ }
+ poll_interval = PHY_POLL_LINK_OFF;
+ }
+
+ schedule_delayed_work(&p->link_work, poll_interval);
+}
+
+struct xgene_mac_ops xgene_sgmac_ops = {
+ .init = xgene_sgmac_init,
+ .reset = xgene_sgmac_reset,
+ .rx_enable = xgene_sgmac_rx_enable,
+ .tx_enable = xgene_sgmac_tx_enable,
+ .rx_disable = xgene_sgmac_rx_disable,
+ .tx_disable = xgene_sgmac_tx_disable,
+ .set_mac_addr = xgene_sgmac_set_mac_addr,
+ .link_state = xgene_enet_link_state
+};
+
+struct xgene_port_ops xgene_sgport_ops = {
+ .reset = xgene_enet_reset,
+ .cle_bypass = xgene_enet_cle_bypass,
+ .shutdown = xgene_enet_shutdown
+};
--- /dev/null
+/* Applied Micro X-Gene SoC Ethernet Driver
+ *
+ * Copyright (c) 2014, Applied Micro Circuits Corporation
+ * Authors: Iyappan Subramanian <isubramanian@apm.com>
+ * Keyur Chudgar <kchudgar@apm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __XGENE_ENET_SGMAC_H__
+#define __XGENE_ENET_SGMAC_H__
+
+#define PHY_ADDR(src) (((src)<<8) & GENMASK(12, 8))
+#define REG_ADDR(src) ((src) & GENMASK(4, 0))
+#define PHY_CONTROL(src) ((src) & GENMASK(15, 0))
+#define INT_PHY_ADDR 0x1e
+#define SGMII_TBI_CONTROL_ADDR 0x44
+#define SGMII_CONTROL_ADDR 0x00
+#define SGMII_STATUS_ADDR 0x04
+#define SGMII_BASE_PAGE_ABILITY_ADDR 0x14
+#define AUTO_NEG_COMPLETE BIT(5)
+#define LINK_STATUS BIT(2)
+#define LINK_UP BIT(15)
+#define MPA_IDLE_WITH_QMI_EMPTY BIT(12)
+#define SG_RX_DV_GATE_REG_0_ADDR 0x0dfc
+
+extern struct xgene_mac_ops xgene_sgmac_ops;
+extern struct xgene_port_ops xgene_sgport_ops;
+
+#endif /* __XGENE_ENET_SGMAC_H__ */
clk_disable_unprepare(pdata->clk);
}
-void xgene_enet_link_state(struct work_struct *work)
+static void xgene_enet_link_state(struct work_struct *work)
{
struct xgene_enet_pdata *pdata = container_of(to_delayed_work(work),
struct xgene_enet_pdata, link_work);
.rx_disable = xgene_xgmac_rx_disable,
.tx_disable = xgene_xgmac_tx_disable,
.set_mac_addr = xgene_xgmac_set_mac_addr,
+ .link_state = xgene_enet_link_state
};
struct xgene_port_ops xgene_xgport_ops = {
#define XG_ENET_SPARE_CFG_REG_1_ADDR 0x0410
#define XGENET_RX_DV_GATE_REG_0_ADDR 0x0804
-#define PHY_POLL_LINK_ON (10 * HZ)
-#define PHY_POLL_LINK_OFF (PHY_POLL_LINK_ON / 5)
-
-void xgene_enet_link_state(struct work_struct *work);
extern struct xgene_mac_ops xgene_xgmac_ops;
extern struct xgene_port_ops xgene_xgport_ops;
/* Sync BD data before updating mailbox */
wmb();
- /* Packets are ready, update Tx producer idx local and on card. */
- tw32_tx_mbox(tnapi->prodmbox, entry);
-
tnapi->tx_prod = entry;
if (unlikely(tg3_tx_avail(tnapi) <= (MAX_SKB_FRAGS + 1))) {
netif_tx_stop_queue(txq);
netif_tx_wake_queue(txq);
}
- mmiowb();
+ if (!skb->xmit_more || netif_xmit_stopped(txq)) {
+ /* Packets are ready, update Tx producer idx on card. */
+ tw32_tx_mbox(tnapi->prodmbox, entry);
+ mmiowb();
+ }
+
return NETDEV_TX_OK;
dma_error:
len = (vec == nvecs) ?
last_fraglen : unmap->vector.len;
+ skb->truesize += unmap->vector.len;
totlen += len;
skb_fill_page_desc(skb, skb_shinfo(skb)->nr_frags,
skb->len += totlen;
skb->data_len += totlen;
- skb->truesize += totlen;
}
static inline void
config CHELSIO_T4
tristate "Chelsio Communications T4/T5 Ethernet support"
- depends on PCI
+ depends on PCI && (IPV6 || IPV6=n)
select FW_LOADER
select MDIO
---help---
int t4_seeprom_wp(struct adapter *adapter, bool enable);
int get_vpd_params(struct adapter *adapter, struct vpd_params *p);
int t4_load_fw(struct adapter *adapter, const u8 *fw_data, unsigned int size);
+int t4_fw_upgrade(struct adapter *adap, unsigned int mbox,
+ const u8 *fw_data, unsigned int size, int force);
unsigned int t4_flash_cfg_addr(struct adapter *adapter);
int t4_get_fw_version(struct adapter *adapter, u32 *vers);
int t4_get_tp_version(struct adapter *adapter, u32 *vers);
int ret;
const struct firmware *fw;
struct adapter *adap = netdev2adap(netdev);
+ unsigned int mbox = FW_PCIE_FW_MASTER_MASK + 1;
ef->data[sizeof(ef->data) - 1] = '\0';
ret = request_firmware(&fw, ef->data, adap->pdev_dev);
if (ret < 0)
return ret;
- ret = t4_load_fw(adap, fw->data, fw->size);
+ /* If the adapter has been fully initialized then we'll go ahead and
+ * try to get the firmware's cooperation in upgrading to the new
+ * firmware image otherwise we'll try to do the entire job from the
+ * host ... and we always "force" the operation in this path.
+ */
+ if (adap->flags & FULL_INIT_DONE)
+ mbox = adap->mbox;
+
+ ret = t4_fw_upgrade(adap, mbox, fw->data, fw->size, 1);
release_firmware(fw);
if (!ret)
- dev_info(adap->pdev_dev, "loaded firmware %s\n", ef->data);
+ dev_info(adap->pdev_dev, "loaded firmware %s,"
+ " reload cxgb4 driver\n", ef->data);
return ret;
}
* success (true) if it belongs otherwise failure (false).
* Called with rcu_read_lock() held.
*/
+#if IS_ENABLED(CONFIG_IPV6)
static bool cxgb4_netdev(const struct net_device *netdev)
{
struct adapter *adap;
return ret;
/* Parse all bond and vlan devices layered on top of the physical dev */
+ root_dev = netdev_master_upper_dev_get_rcu(dev);
+ if (root_dev) {
+ ret = update_dev_clip(root_dev, dev);
+ if (ret)
+ return ret;
+ }
+
for (i = 0; i < VLAN_N_VID; i++) {
root_dev = __vlan_find_dev_deep_rcu(dev, htons(ETH_P_8021Q), i);
if (!root_dev)
}
rcu_read_unlock();
}
+#endif /* IS_ENABLED(CONFIG_IPV6) */
/**
* cxgb_up - enable the adapter
t4_intr_enable(adap);
adap->flags |= FULL_INIT_DONE;
notify_ulds(adap, CXGB4_STATE_UP);
+#if IS_ENABLED(CONFIG_IPV6)
update_clip(adap);
+#endif
out:
return err;
irq_err:
if (ret < 0)
debugfs_remove(cxgb4_debugfs_root);
+#if IS_ENABLED(CONFIG_IPV6)
register_inet6addr_notifier(&cxgb4_inet6addr_notifier);
+#endif
return ret;
}
static void __exit cxgb4_cleanup_module(void)
{
+#if IS_ENABLED(CONFIG_IPV6)
unregister_inet6addr_notifier(&cxgb4_inet6addr_notifier);
+#endif
pci_unregister_driver(&cxgb4_driver);
debugfs_remove(cxgb4_debugfs_root); /* NULL ok */
}
#include "t4_regs.h"
#include "t4fw_api.h"
-static int t4_fw_upgrade(struct adapter *adap, unsigned int mbox,
- const u8 *fw_data, unsigned int size, int force);
/**
* t4_wait_op_done_val - wait until an operation is completed
* @adapter: the adapter performing the operation
* positive errno indicates that the adapter is ~probably~ intact, a
* negative errno indicates that things are looking bad ...
*/
-static int t4_fw_upgrade(struct adapter *adap, unsigned int mbox,
- const u8 *fw_data, unsigned int size, int force)
+int t4_fw_upgrade(struct adapter *adap, unsigned int mbox,
+ const u8 *fw_data, unsigned int size, int force)
{
const struct fw_hdr *fw_hdr = (const struct fw_hdr *)fw_data;
int reset, ret;
#define FEC_VLAN_TAG_LEN 0x04
#define FEC_ETHTYPE_LEN 0x02
+/* Controller is ENET-MAC */
+#define FEC_QUIRK_ENET_MAC (1 << 0)
+/* Controller needs driver to swap frame */
+#define FEC_QUIRK_SWAP_FRAME (1 << 1)
+/* Controller uses gasket */
+#define FEC_QUIRK_USE_GASKET (1 << 2)
+/* Controller has GBIT support */
+#define FEC_QUIRK_HAS_GBIT (1 << 3)
+/* Controller has extend desc buffer */
+#define FEC_QUIRK_HAS_BUFDESC_EX (1 << 4)
+/* Controller has hardware checksum support */
+#define FEC_QUIRK_HAS_CSUM (1 << 5)
+/* Controller has hardware vlan support */
+#define FEC_QUIRK_HAS_VLAN (1 << 6)
+/* ENET IP errata ERR006358
+ *
+ * If the ready bit in the transmit buffer descriptor (TxBD[R]) is previously
+ * detected as not set during a prior frame transmission, then the
+ * ENET_TDAR[TDAR] bit is cleared at a later time, even if additional TxBDs
+ * were added to the ring and the ENET_TDAR[TDAR] bit is set. This results in
+ * frames not being transmitted until there is a 0-to-1 transition on
+ * ENET_TDAR[TDAR].
+ */
+#define FEC_QUIRK_ERR006358 (1 << 7)
+/* ENET IP hw AVB
+ *
+ * i.MX6SX ENET IP add Audio Video Bridging (AVB) feature support.
+ * - Two class indicators on receive with configurable priority
+ * - Two class indicators and line speed timer on transmit allowing
+ * implementation class credit based shapers externally
+ * - Additional DMA registers provisioned to allow managing up to 3
+ * independent rings
+ */
+#define FEC_QUIRK_HAS_AVB (1 << 8)
+/* There is a TDAR race condition for mutliQ when the software sets TDAR
+ * and the UDMA clears TDAR simultaneously or in a small window (2-4 cycles).
+ * This will cause the udma_tx and udma_tx_arbiter state machines to hang.
+ * The issue exist at i.MX6SX enet IP.
+ */
+#define FEC_QUIRK_ERR007885 (1 << 9)
+/* ENET Block Guide/ Chapter for the iMX6SX (PELE) address one issue:
+ * After set ENET_ATCR[Capture], there need some time cycles before the counter
+ * value is capture in the register clock domain.
+ * The wait-time-cycles is at least 6 clock cycles of the slower clock between
+ * the register clock and the 1588 clock. The 1588 ts_clk is fixed to 25Mhz,
+ * register clock is 66Mhz, so the wait-time-cycles must be greater than 240ns
+ * (40ns * 6).
+ */
+#define FEC_QUIRK_BUG_CAPTURE (1 << 10)
+
struct fec_enet_priv_tx_q {
int index;
unsigned char *tx_bounce[TX_RING_SIZE];
unsigned int itr_clk_rate;
u32 rx_copybreak;
+
+ /* ptp clock period in ns*/
+ unsigned int ptp_inc;
+
+ /* pps */
+ int pps_channel;
+ unsigned int reload_period;
+ int pps_enable;
+ unsigned int next_counter;
};
void fec_ptp_init(struct platform_device *pdev);
void fec_ptp_start_cyclecounter(struct net_device *ndev);
int fec_ptp_set(struct net_device *ndev, struct ifreq *ifr);
int fec_ptp_get(struct net_device *ndev, struct ifreq *ifr);
+uint fec_ptp_check_pps_event(struct fec_enet_private *fep);
/****************************************************************************/
#endif /* FEC_H */
#define FEC_ENET_RAFL_V 0x8
#define FEC_ENET_OPD_V 0xFFF0
-/* Controller is ENET-MAC */
-#define FEC_QUIRK_ENET_MAC (1 << 0)
-/* Controller needs driver to swap frame */
-#define FEC_QUIRK_SWAP_FRAME (1 << 1)
-/* Controller uses gasket */
-#define FEC_QUIRK_USE_GASKET (1 << 2)
-/* Controller has GBIT support */
-#define FEC_QUIRK_HAS_GBIT (1 << 3)
-/* Controller has extend desc buffer */
-#define FEC_QUIRK_HAS_BUFDESC_EX (1 << 4)
-/* Controller has hardware checksum support */
-#define FEC_QUIRK_HAS_CSUM (1 << 5)
-/* Controller has hardware vlan support */
-#define FEC_QUIRK_HAS_VLAN (1 << 6)
-/* ENET IP errata ERR006358
- *
- * If the ready bit in the transmit buffer descriptor (TxBD[R]) is previously
- * detected as not set during a prior frame transmission, then the
- * ENET_TDAR[TDAR] bit is cleared at a later time, even if additional TxBDs
- * were added to the ring and the ENET_TDAR[TDAR] bit is set. This results in
- * frames not being transmitted until there is a 0-to-1 transition on
- * ENET_TDAR[TDAR].
- */
-#define FEC_QUIRK_ERR006358 (1 << 7)
-/* ENET IP hw AVB
- *
- * i.MX6SX ENET IP add Audio Video Bridging (AVB) feature support.
- * - Two class indicators on receive with configurable priority
- * - Two class indicators and line speed timer on transmit allowing
- * implementation class credit based shapers externally
- * - Additional DMA registers provisioned to allow managing up to 3
- * independent rings
- */
-#define FEC_QUIRK_HAS_AVB (1 << 8)
-/* There is a TDAR race condition for mutliQ when the software sets TDAR
- * and the UDMA clears TDAR simultaneously or in a small window (2-4 cycles).
- * This will cause the udma_tx and udma_tx_arbiter state machines to hang.
- * The issue exist at i.MX6SX enet IP.
- */
-#define FEC_QUIRK_ERR007885 (1 << 9)
-
static struct platform_device_id fec_devtype[] = {
{
/* keep it for coldfire */
.driver_data = FEC_QUIRK_ENET_MAC | FEC_QUIRK_HAS_GBIT |
FEC_QUIRK_HAS_BUFDESC_EX | FEC_QUIRK_HAS_CSUM |
FEC_QUIRK_HAS_VLAN | FEC_QUIRK_HAS_AVB |
- FEC_QUIRK_ERR007885,
+ FEC_QUIRK_ERR007885 | FEC_QUIRK_BUG_CAPTURE,
}, {
/* sentinel */
}
complete(&fep->mdio_done);
}
+ fec_ptp_check_pps_event(fep);
+
return ret;
}
#endif
#define FEATURES_NEED_QUIESCE NETIF_F_RXCSUM
-
-static int fec_set_features(struct net_device *netdev,
+static inline void fec_enet_set_netdev_features(struct net_device *netdev,
netdev_features_t features)
{
struct fec_enet_private *fep = netdev_priv(netdev);
netdev_features_t changed = features ^ netdev->features;
- /* Quiesce the device if necessary */
- if (netif_running(netdev) && changed & FEATURES_NEED_QUIESCE) {
- napi_disable(&fep->napi);
- netif_tx_lock_bh(netdev);
- fec_stop(netdev);
- }
-
netdev->features = features;
/* Receive checksum has been changed */
else
fep->csum_flags &= ~FLAG_RX_CSUM_ENABLED;
}
+}
+
+static int fec_set_features(struct net_device *netdev,
+ netdev_features_t features)
+{
+ struct fec_enet_private *fep = netdev_priv(netdev);
+ netdev_features_t changed = features ^ netdev->features;
- /* Resume the device after updates */
if (netif_running(netdev) && changed & FEATURES_NEED_QUIESCE) {
+ napi_disable(&fep->napi);
+ netif_tx_lock_bh(netdev);
+ fec_stop(netdev);
+ fec_enet_set_netdev_features(netdev, features);
fec_restart(netdev);
netif_tx_wake_all_queues(netdev);
netif_tx_unlock_bh(netdev);
napi_enable(&fep->napi);
+ } else {
+ fec_enet_set_netdev_features(netdev, features);
}
return 0;
#define FEC_T_INC_CORR_MASK 0x00007f00
#define FEC_T_INC_CORR_OFFSET 8
+#define FEC_T_CTRL_PINPER 0x00000080
+#define FEC_T_TF0_MASK 0x00000001
+#define FEC_T_TF0_OFFSET 0
+#define FEC_T_TF1_MASK 0x00000002
+#define FEC_T_TF1_OFFSET 1
+#define FEC_T_TF2_MASK 0x00000004
+#define FEC_T_TF2_OFFSET 2
+#define FEC_T_TF3_MASK 0x00000008
+#define FEC_T_TF3_OFFSET 3
+#define FEC_T_TDRE_MASK 0x00000001
+#define FEC_T_TDRE_OFFSET 0
+#define FEC_T_TMODE_MASK 0x0000003C
+#define FEC_T_TMODE_OFFSET 2
+#define FEC_T_TIE_MASK 0x00000040
+#define FEC_T_TIE_OFFSET 6
+#define FEC_T_TF_MASK 0x00000080
+#define FEC_T_TF_OFFSET 7
+
#define FEC_ATIME_CTRL 0x400
#define FEC_ATIME 0x404
#define FEC_ATIME_EVT_OFFSET 0x408
#define FEC_ATIME_INC 0x414
#define FEC_TS_TIMESTAMP 0x418
+#define FEC_TGSR 0x604
+#define FEC_TCSR(n) (0x608 + n * 0x08)
+#define FEC_TCCR(n) (0x60C + n * 0x08)
+#define MAX_TIMER_CHANNEL 3
+#define FEC_TMODE_TOGGLE 0x05
+#define FEC_HIGH_PULSE 0x0F
+
#define FEC_CC_MULT (1 << 31)
+#define FEC_COUNTER_PERIOD (1 << 31)
+#define PPS_OUPUT_RELOAD_PERIOD NSEC_PER_SEC
+#define FEC_CHANNLE_0 0
+#define DEFAULT_PPS_CHANNEL FEC_CHANNLE_0
+
+/**
+ * fec_ptp_enable_pps
+ * @fep: the fec_enet_private structure handle
+ * @enable: enable the channel pps output
+ *
+ * This function enble the PPS ouput on the timer channel.
+ */
+static int fec_ptp_enable_pps(struct fec_enet_private *fep, uint enable)
+{
+ unsigned long flags;
+ u32 val, tempval;
+ int inc;
+ struct timespec ts;
+ u64 ns;
+ u32 remainder;
+ val = 0;
+
+ if (!(fep->hwts_tx_en || fep->hwts_rx_en)) {
+ dev_err(&fep->pdev->dev, "No ptp stack is running\n");
+ return -EINVAL;
+ }
+
+ if (fep->pps_enable == enable)
+ return 0;
+
+ fep->pps_channel = DEFAULT_PPS_CHANNEL;
+ fep->reload_period = PPS_OUPUT_RELOAD_PERIOD;
+ inc = fep->ptp_inc;
+
+ spin_lock_irqsave(&fep->tmreg_lock, flags);
+
+ if (enable) {
+ /* clear capture or output compare interrupt status if have.
+ */
+ writel(FEC_T_TF_MASK, fep->hwp + FEC_TCSR(fep->pps_channel));
+
+ /* It is recommended to doulbe check the TMODE field in the
+ * TCSR register to be cleared before the first compare counter
+ * is written into TCCR register. Just add a double check.
+ */
+ val = readl(fep->hwp + FEC_TCSR(fep->pps_channel));
+ do {
+ val &= ~(FEC_T_TMODE_MASK);
+ writel(val, fep->hwp + FEC_TCSR(fep->pps_channel));
+ val = readl(fep->hwp + FEC_TCSR(fep->pps_channel));
+ } while (val & FEC_T_TMODE_MASK);
+
+ /* Dummy read counter to update the counter */
+ timecounter_read(&fep->tc);
+ /* We want to find the first compare event in the next
+ * second point. So we need to know what the ptp time
+ * is now and how many nanoseconds is ahead to get next second.
+ * The remaining nanosecond ahead before the next second would be
+ * NSEC_PER_SEC - ts.tv_nsec. Add the remaining nanoseconds
+ * to current timer would be next second.
+ */
+ tempval = readl(fep->hwp + FEC_ATIME_CTRL);
+ tempval |= FEC_T_CTRL_CAPTURE;
+ writel(tempval, fep->hwp + FEC_ATIME_CTRL);
+
+ tempval = readl(fep->hwp + FEC_ATIME);
+ /* Convert the ptp local counter to 1588 timestamp */
+ ns = timecounter_cyc2time(&fep->tc, tempval);
+ ts.tv_sec = div_u64_rem(ns, 1000000000ULL, &remainder);
+ ts.tv_nsec = remainder;
+
+ /* The tempval is less than 3 seconds, and so val is less than
+ * 4 seconds. No overflow for 32bit calculation.
+ */
+ val = NSEC_PER_SEC - (u32)ts.tv_nsec + tempval;
+
+ /* Need to consider the situation that the current time is
+ * very close to the second point, which means NSEC_PER_SEC
+ * - ts.tv_nsec is close to be zero(For example 20ns); Since the timer
+ * is still running when we calculate the first compare event, it is
+ * possible that the remaining nanoseonds run out before the compare
+ * counter is calculated and written into TCCR register. To avoid
+ * this possibility, we will set the compare event to be the next
+ * of next second. The current setting is 31-bit timer and wrap
+ * around over 2 seconds. So it is okay to set the next of next
+ * seond for the timer.
+ */
+ val += NSEC_PER_SEC;
+
+ /* We add (2 * NSEC_PER_SEC - (u32)ts.tv_nsec) to current
+ * ptp counter, which maybe cause 32-bit wrap. Since the
+ * (NSEC_PER_SEC - (u32)ts.tv_nsec) is less than 2 second.
+ * We can ensure the wrap will not cause issue. If the offset
+ * is bigger than fep->cc.mask would be a error.
+ */
+ val &= fep->cc.mask;
+ writel(val, fep->hwp + FEC_TCCR(fep->pps_channel));
+
+ /* Calculate the second the compare event timestamp */
+ fep->next_counter = (val + fep->reload_period) & fep->cc.mask;
+
+ /* * Enable compare event when overflow */
+ val = readl(fep->hwp + FEC_ATIME_CTRL);
+ val |= FEC_T_CTRL_PINPER;
+ writel(val, fep->hwp + FEC_ATIME_CTRL);
+
+ /* Compare channel setting. */
+ val = readl(fep->hwp + FEC_TCSR(fep->pps_channel));
+ val |= (1 << FEC_T_TF_OFFSET | 1 << FEC_T_TIE_OFFSET);
+ val &= ~(1 << FEC_T_TDRE_OFFSET);
+ val &= ~(FEC_T_TMODE_MASK);
+ val |= (FEC_HIGH_PULSE << FEC_T_TMODE_OFFSET);
+ writel(val, fep->hwp + FEC_TCSR(fep->pps_channel));
+
+ /* Write the second compare event timestamp and calculate
+ * the third timestamp. Refer the TCCR register detail in the spec.
+ */
+ writel(fep->next_counter, fep->hwp + FEC_TCCR(fep->pps_channel));
+ fep->next_counter = (fep->next_counter + fep->reload_period) & fep->cc.mask;
+ } else {
+ writel(0, fep->hwp + FEC_TCSR(fep->pps_channel));
+ }
+
+ fep->pps_enable = enable;
+ spin_unlock_irqrestore(&fep->tmreg_lock, flags);
+
+ return 0;
+}
+
/**
* fec_ptp_read - read raw cycle counter (to be used by time counter)
* @cc: the cyclecounter structure
{
struct fec_enet_private *fep =
container_of(cc, struct fec_enet_private, cc);
+ const struct platform_device_id *id_entry =
+ platform_get_device_id(fep->pdev);
u32 tempval;
tempval = readl(fep->hwp + FEC_ATIME_CTRL);
tempval |= FEC_T_CTRL_CAPTURE;
writel(tempval, fep->hwp + FEC_ATIME_CTRL);
+ if (id_entry->driver_data & FEC_QUIRK_BUG_CAPTURE)
+ udelay(1);
+
return readl(fep->hwp + FEC_ATIME);
}
/* 1ns counter */
writel(inc << FEC_T_INC_OFFSET, fep->hwp + FEC_ATIME_INC);
- /* use free running count */
- writel(0, fep->hwp + FEC_ATIME_EVT_PERIOD);
+ /* use 31-bit timer counter */
+ writel(FEC_COUNTER_PERIOD, fep->hwp + FEC_ATIME_EVT_PERIOD);
- writel(FEC_T_CTRL_ENABLE, fep->hwp + FEC_ATIME_CTRL);
+ writel(FEC_T_CTRL_ENABLE | FEC_T_CTRL_PERIOD_RST,
+ fep->hwp + FEC_ATIME_CTRL);
memset(&fep->cc, 0, sizeof(fep->cc));
fep->cc.read = fec_ptp_read;
- fep->cc.mask = CLOCKSOURCE_MASK(32);
+ fep->cc.mask = CLOCKSOURCE_MASK(31);
fep->cc.shift = 31;
fep->cc.mult = FEC_CC_MULT;
*/
static int fec_ptp_adjfreq(struct ptp_clock_info *ptp, s32 ppb)
{
- u64 diff;
unsigned long flags;
int neg_adj = 0;
- u32 mult = FEC_CC_MULT;
+ u32 i, tmp;
+ u32 corr_inc, corr_period;
+ u32 corr_ns;
+ u64 lhs, rhs;
struct fec_enet_private *fep =
container_of(ptp, struct fec_enet_private, ptp_caps);
+ if (ppb == 0)
+ return 0;
+
if (ppb < 0) {
ppb = -ppb;
neg_adj = 1;
}
- diff = mult;
- diff *= ppb;
- diff = div_u64(diff, 1000000000ULL);
+ /* In theory, corr_inc/corr_period = ppb/NSEC_PER_SEC;
+ * Try to find the corr_inc between 1 to fep->ptp_inc to
+ * meet adjustment requirement.
+ */
+ lhs = NSEC_PER_SEC;
+ rhs = (u64)ppb * (u64)fep->ptp_inc;
+ for (i = 1; i <= fep->ptp_inc; i++) {
+ if (lhs >= rhs) {
+ corr_inc = i;
+ corr_period = div_u64(lhs, rhs);
+ break;
+ }
+ lhs += NSEC_PER_SEC;
+ }
+ /* Not found? Set it to high value - double speed
+ * correct in every clock step.
+ */
+ if (i > fep->ptp_inc) {
+ corr_inc = fep->ptp_inc;
+ corr_period = 1;
+ }
+
+ if (neg_adj)
+ corr_ns = fep->ptp_inc - corr_inc;
+ else
+ corr_ns = fep->ptp_inc + corr_inc;
spin_lock_irqsave(&fep->tmreg_lock, flags);
- /*
- * dummy read to set cycle_last in tc to now.
- * So use adjusted mult to calculate when next call
- * timercounter_read.
- */
- timecounter_read(&fep->tc);
- fep->cc.mult = neg_adj ? mult - diff : mult + diff;
+ tmp = readl(fep->hwp + FEC_ATIME_INC) & FEC_T_INC_MASK;
+ tmp |= corr_ns << FEC_T_INC_CORR_OFFSET;
+ writel(tmp, fep->hwp + FEC_ATIME_INC);
+ writel(corr_period, fep->hwp + FEC_ATIME_CORR);
+ /* dummy read to update the timer. */
+ timecounter_read(&fep->tc);
spin_unlock_irqrestore(&fep->tmreg_lock, flags);
container_of(ptp, struct fec_enet_private, ptp_caps);
unsigned long flags;
u64 now;
+ u32 counter;
spin_lock_irqsave(&fep->tmreg_lock, flags);
now = timecounter_read(&fep->tc);
now += delta;
+ /* Get the timer value based on adjusted timestamp.
+ * Update the counter with the masked value.
+ */
+ counter = now & fep->cc.mask;
+ writel(counter, fep->hwp + FEC_ATIME);
+
/* reset the timecounter */
timecounter_init(&fep->tc, &fep->cc, now);
u64 ns;
unsigned long flags;
+ u32 counter;
mutex_lock(&fep->ptp_clk_mutex);
/* Check the ptp clock */
ns = ts->tv_sec * 1000000000ULL;
ns += ts->tv_nsec;
+ /* Get the timer value based on timestamp.
+ * Update the counter with the masked value.
+ */
+ counter = ns & fep->cc.mask;
spin_lock_irqsave(&fep->tmreg_lock, flags);
+ writel(counter, fep->hwp + FEC_ATIME);
timecounter_init(&fep->tc, &fep->cc, ns);
spin_unlock_irqrestore(&fep->tmreg_lock, flags);
mutex_unlock(&fep->ptp_clk_mutex);
static int fec_ptp_enable(struct ptp_clock_info *ptp,
struct ptp_clock_request *rq, int on)
{
+ struct fec_enet_private *fep =
+ container_of(ptp, struct fec_enet_private, ptp_caps);
+ int ret = 0;
+
+ if (rq->type == PTP_CLK_REQ_PPS) {
+ ret = fec_ptp_enable_pps(fep, on);
+
+ return ret;
+ }
return -EOPNOTSUPP;
}
fep->ptp_caps.n_ext_ts = 0;
fep->ptp_caps.n_per_out = 0;
fep->ptp_caps.n_pins = 0;
- fep->ptp_caps.pps = 0;
+ fep->ptp_caps.pps = 1;
fep->ptp_caps.adjfreq = fec_ptp_adjfreq;
fep->ptp_caps.adjtime = fec_ptp_adjtime;
fep->ptp_caps.gettime = fec_ptp_gettime;
fep->ptp_caps.enable = fec_ptp_enable;
fep->cycle_speed = clk_get_rate(fep->clk_ptp);
+ fep->ptp_inc = NSEC_PER_SEC / fep->cycle_speed;
spin_lock_init(&fep->tmreg_lock);
schedule_delayed_work(&fep->time_keep, HZ);
}
+
+/**
+ * fec_ptp_check_pps_event
+ * @fep: the fec_enet_private structure handle
+ *
+ * This function check the pps event and reload the timer compare counter.
+ */
+uint fec_ptp_check_pps_event(struct fec_enet_private *fep)
+{
+ u32 val;
+ u8 channel = fep->pps_channel;
+ struct ptp_clock_event event;
+
+ val = readl(fep->hwp + FEC_TCSR(channel));
+ if (val & FEC_T_TF_MASK) {
+ /* Write the next next compare(not the next according the spec)
+ * value to the register
+ */
+ writel(fep->next_counter, fep->hwp + FEC_TCCR(channel));
+ do {
+ writel(val, fep->hwp + FEC_TCSR(channel));
+ } while (readl(fep->hwp + FEC_TCSR(channel)) & FEC_T_TF_MASK);
+
+ /* Update the counter; */
+ fep->next_counter = (fep->next_counter + fep->reload_period) & fep->cc.mask;
+
+ event.type = PTP_CLOCK_PPS;
+ ptp_clock_event(fep->ptp_clock, &event);
+ return 1;
+ }
+
+ return 0;
+}
static void gfar_rx_buff_size_config(struct gfar_private *priv)
{
- int frame_size = priv->ndev->mtu + ETH_HLEN;
+ int frame_size = priv->ndev->mtu + ETH_HLEN + ETH_FCS_LEN;
/* set this when rx hw offload (TOE) functions are being used */
priv->uses_rxfcb = 0;
To compile this driver as a module, choose M here. The module
will be called fm10k. MSI-X interrupt support is required
+config FM10K_VXLAN
+ bool "Virtual eXtensible Local Area Network Support"
+ default n
+ depends on FM10K && VXLAN && !(FM10K=y && VXLAN=m)
+ ---help---
+ This allows one to create VXLAN virtual interfaces that provide
+ Layer 2 Networks over Layer 3 Networks. VXLAN is often used
+ to tunnel virtual network infrastructure in virtualized environments.
+ Say Y here if you want to use Virtual eXtensible Local Area Network
+ (VXLAN) in the driver.
+
endif # NET_VENDOR_INTEL
return i == tx_ring->count;
}
+static int __fm10k_maybe_stop_tx(struct fm10k_ring *tx_ring, u16 size)
+{
+ netif_stop_subqueue(tx_ring->netdev, tx_ring->queue_index);
+
+ smp_mb();
+
+ /* We need to check again in a case another CPU has just
+ * made room available. */
+ if (likely(fm10k_desc_unused(tx_ring) < size))
+ return -EBUSY;
+
+ /* A reprieve! - use start_queue because it doesn't call schedule */
+ netif_start_subqueue(tx_ring->netdev, tx_ring->queue_index);
+ ++tx_ring->tx_stats.restart_queue;
+ return 0;
+}
+
+static inline int fm10k_maybe_stop_tx(struct fm10k_ring *tx_ring, u16 size)
+{
+ if (likely(fm10k_desc_unused(tx_ring) >= size))
+ return 0;
+ return __fm10k_maybe_stop_tx(tx_ring, size);
+}
+
static void fm10k_tx_map(struct fm10k_ring *tx_ring,
struct fm10k_tx_buffer *first)
{
tx_ring->next_to_use = i;
+ /* Make sure there is space in the ring for the next send. */
+ fm10k_maybe_stop_tx(tx_ring, DESC_NEEDED);
+
/* notify HW of packet */
- writel(i, tx_ring->tail);
+ if (netif_xmit_stopped(txring_txq(tx_ring)) || !skb->xmit_more) {
+ writel(i, tx_ring->tail);
- /* we need this if more than one processor can write to our tail
- * at a time, it synchronizes IO on IA64/Altix systems
- */
- mmiowb();
+ /* we need this if more than one processor can write to our tail
+ * at a time, it synchronizes IO on IA64/Altix systems
+ */
+ mmiowb();
+ }
return;
dma_error:
tx_ring->next_to_use = i;
}
-static int __fm10k_maybe_stop_tx(struct fm10k_ring *tx_ring, u16 size)
-{
- netif_stop_subqueue(tx_ring->netdev, tx_ring->queue_index);
-
- smp_mb();
-
- /* We need to check again in a case another CPU has just
- * made room available. */
- if (likely(fm10k_desc_unused(tx_ring) < size))
- return -EBUSY;
-
- /* A reprieve! - use start_queue because it doesn't call schedule */
- netif_start_subqueue(tx_ring->netdev, tx_ring->queue_index);
- ++tx_ring->tx_stats.restart_queue;
- return 0;
-}
-
-static inline int fm10k_maybe_stop_tx(struct fm10k_ring *tx_ring, u16 size)
-{
- if (likely(fm10k_desc_unused(tx_ring) >= size))
- return 0;
- return __fm10k_maybe_stop_tx(tx_ring, size);
-}
-
netdev_tx_t fm10k_xmit_frame_ring(struct sk_buff *skb,
struct fm10k_ring *tx_ring)
{
fm10k_tx_map(tx_ring, first);
- fm10k_maybe_stop_tx(tx_ring, DESC_NEEDED);
-
return NETDEV_TX_OK;
out_drop:
#include "fm10k.h"
#include <linux/vmalloc.h>
-#if IS_ENABLED(CONFIG_VXLAN)
+#if IS_ENABLED(CONFIG_FM10K_VXLAN)
#include <net/vxlan.h>
-#endif /* CONFIG_VXLAN */
+#endif /* CONFIG_FM10K_VXLAN */
/**
* fm10k_setup_tx_resources - allocate Tx resources (Descriptors)
if (err)
goto err_set_queues;
-#if IS_ENABLED(CONFIG_VXLAN)
+#if IS_ENABLED(CONFIG_FM10K_VXLAN)
/* update VXLAN port configuration */
vxlan_get_rx_port(netdev);
if (!(netdev->flags & IFF_PROMISC)) {
err = hw->mac.ops.update_vlan(hw, vid, 0, set);
if (err)
- return err;
+ goto err_out;
}
/* update our base MAC address */
err = hw->mac.ops.update_uc_addr(hw, interface->glort, hw->mac.addr,
vid, set, 0);
if (err)
- return err;
+ goto err_out;
/* set vid prior to syncing/unsyncing the VLAN */
interface->vid = vid + (set ? VLAN_N_VID : 0);
__dev_uc_unsync(netdev, fm10k_uc_vlan_unsync);
__dev_mc_unsync(netdev, fm10k_mc_vlan_unsync);
+err_out:
fm10k_mbx_unlock(interface);
- return 0;
+ return err;
}
static int fm10k_vlan_rx_add_vid(struct net_device *netdev,
netif_tx_start_all_queues(interface->netdev);
/* kick off the service timer */
+ hw->mac.get_host_state = 1;
mod_timer(&interface->service_timer, jiffies);
}
struct ixgbe_hw *hw = &adapter->hw;
u32 regval;
+ if (vf >= adapter->num_vfs)
+ return -EINVAL;
+
adapter->vfinfo[vf].spoofchk_enabled = setting;
regval = IXGBE_READ_REG(hw, IXGBE_PFVFSPOOF(vf_target_reg));
{QLCNIC_CMD_CREATE_RX_CTX, 4, 1},
{QLCNIC_CMD_DESTROY_RX_CTX, 2, 1},
{QLCNIC_CMD_CREATE_TX_CTX, 4, 1},
- {QLCNIC_CMD_DESTROY_TX_CTX, 2, 1},
+ {QLCNIC_CMD_DESTROY_TX_CTX, 3, 1},
{QLCNIC_CMD_INTRPT_TEST, 4, 1},
{QLCNIC_CMD_SET_MTU, 4, 1},
{QLCNIC_CMD_READ_PHY, 4, 2},
{QLCNIC_CMD_CONFIGURE_ESWITCH, 4, 1},
{QLCNIC_CMD_GET_MAC_STATS, 4, 1},
{QLCNIC_CMD_GET_ESWITCH_PORT_CONFIG, 4, 3},
- {QLCNIC_CMD_GET_ESWITCH_STATS, 5, 1},
+ {QLCNIC_CMD_GET_ESWITCH_STATS, 4, 1},
{QLCNIC_CMD_CONFIG_PORT, 4, 1},
{QLCNIC_CMD_TEMP_SIZE, 4, 4},
{QLCNIC_CMD_GET_TEMP_HDR, 4, 1},
}
QLCWR32(adapter, QLCNIC_SIGN_CRB_OFFSET, signature);
- for (i = 1; i < QLCNIC_CDRP_MAX_ARGS; i++)
+ for (i = 1; i < cmd->req.num; i++)
QLCWR32(adapter, QLCNIC_CDRP_ARG(i), cmd->req.arg[i]);
QLCWR32(adapter, QLCNIC_CDRP_CRB_OFFSET,
QLCNIC_CDRP_FORM_CMD(cmd->req.arg[0]));
return ((efx_qword_t *) (tx_queue->txd.buf.addr)) + index;
}
-/* Report whether the NIC considers this TX queue empty, given the
- * write_count used for the last doorbell push. May return false
- * negative.
+/* Get partner of a TX queue, seen as part of the same net core queue */
+static struct efx_tx_queue *efx_tx_queue_partner(struct efx_tx_queue *tx_queue)
+{
+ if (tx_queue->queue & EFX_TXQ_TYPE_OFFLOAD)
+ return tx_queue - EFX_TXQ_TYPE_OFFLOAD;
+ else
+ return tx_queue + EFX_TXQ_TYPE_OFFLOAD;
+}
+
+/* Report whether this TX queue would be empty for the given write_count.
+ * May return false negative.
*/
static inline bool __efx_nic_tx_is_empty(struct efx_tx_queue *tx_queue,
unsigned int write_count)
return ((empty_read_count ^ write_count) & ~EFX_EMPTY_COUNT_VALID) == 0;
}
-static inline bool efx_nic_tx_is_empty(struct efx_tx_queue *tx_queue)
+/* Decide whether we can use TX PIO, ie. write packet data directly into
+ * a buffer on the device. This can reduce latency at the expense of
+ * throughput, so we only do this if both hardware and software TX rings
+ * are empty. This also ensures that only one packet at a time can be
+ * using the PIO buffer.
+ */
+static inline bool efx_nic_may_tx_pio(struct efx_tx_queue *tx_queue)
{
- return __efx_nic_tx_is_empty(tx_queue, tx_queue->write_count);
+ struct efx_tx_queue *partner = efx_tx_queue_partner(tx_queue);
+ return tx_queue->piobuf &&
+ __efx_nic_tx_is_empty(tx_queue, tx_queue->insert_count) &&
+ __efx_nic_tx_is_empty(partner, partner->insert_count);
}
/* Decide whether to push a TX descriptor to the NIC vs merely writing
* descriptor to an empty queue, but is otherwise pointless. Further,
* Falcon and Siena have hardware bugs (SF bug 33851) that may be
* triggered if we don't check this.
+ * We use the write_count used for the last doorbell push, to get the
+ * NIC's view of the tx queue.
*/
static inline bool efx_nic_may_push_tx_desc(struct efx_tx_queue *tx_queue,
unsigned int write_count)
return max_descs;
}
-/* Get partner of a TX queue, seen as part of the same net core queue */
-static struct efx_tx_queue *efx_tx_queue_partner(struct efx_tx_queue *tx_queue)
-{
- if (tx_queue->queue & EFX_TXQ_TYPE_OFFLOAD)
- return tx_queue - EFX_TXQ_TYPE_OFFLOAD;
- else
- return tx_queue + EFX_TXQ_TYPE_OFFLOAD;
-}
-
static void efx_tx_maybe_stop_queue(struct efx_tx_queue *txq1)
{
/* We need to consider both queues that the net core sees as one */
struct efx_nic *efx = tx_queue->efx;
struct device *dma_dev = &efx->pci_dev->dev;
struct efx_tx_buffer *buffer;
+ unsigned int old_insert_count = tx_queue->insert_count;
skb_frag_t *fragment;
unsigned int len, unmap_len = 0;
dma_addr_t dma_addr, unmap_addr = 0;
unsigned short dma_flags;
int i = 0;
- EFX_BUG_ON_PARANOID(tx_queue->write_count != tx_queue->insert_count);
+ EFX_BUG_ON_PARANOID(tx_queue->write_count > tx_queue->insert_count);
if (skb_shinfo(skb)->gso_size)
return efx_enqueue_skb_tso(tx_queue, skb);
/* Consider using PIO for short packets */
#ifdef EFX_USE_PIO
- if (skb->len <= efx_piobuf_size && tx_queue->piobuf &&
- efx_nic_tx_is_empty(tx_queue) &&
- efx_nic_tx_is_empty(efx_tx_queue_partner(tx_queue))) {
+ if (skb->len <= efx_piobuf_size && !skb->xmit_more &&
+ efx_nic_may_tx_pio(tx_queue)) {
buffer = efx_enqueue_skb_pio(tx_queue, skb);
dma_flags = EFX_TX_BUF_OPTION;
goto finish_packet;
netdev_tx_sent_queue(tx_queue->core_txq, skb->len);
+ efx_tx_maybe_stop_queue(tx_queue);
+
/* Pass off to hardware */
- efx_nic_push_buffers(tx_queue);
+ if (!skb->xmit_more || netif_xmit_stopped(tx_queue->core_txq))
+ efx_nic_push_buffers(tx_queue);
tx_queue->tx_packets++;
- efx_tx_maybe_stop_queue(tx_queue);
-
return NETDEV_TX_OK;
dma_err:
dev_kfree_skb_any(skb);
/* Work backwards until we hit the original insert pointer value */
- while (tx_queue->insert_count != tx_queue->write_count) {
+ while (tx_queue->insert_count != old_insert_count) {
unsigned int pkts_compl = 0, bytes_compl = 0;
--tx_queue->insert_count;
buffer = __efx_tx_queue_get_insert_buffer(tx_queue);
/* Remove buffers put into a tx_queue. None of the buffers must have
* an skb attached.
*/
-static void efx_enqueue_unwind(struct efx_tx_queue *tx_queue)
+static void efx_enqueue_unwind(struct efx_tx_queue *tx_queue,
+ unsigned int insert_count)
{
struct efx_tx_buffer *buffer;
/* Work backwards until we hit the original insert pointer value */
- while (tx_queue->insert_count != tx_queue->write_count) {
+ while (tx_queue->insert_count != insert_count) {
--tx_queue->insert_count;
buffer = __efx_tx_queue_get_insert_buffer(tx_queue);
efx_dequeue_buffer(tx_queue, buffer, NULL, NULL);
struct sk_buff *skb)
{
struct efx_nic *efx = tx_queue->efx;
+ unsigned int old_insert_count = tx_queue->insert_count;
int frag_i, rc;
struct tso_state state;
/* Find the packet protocol and sanity-check it */
state.protocol = efx_tso_check_protocol(skb);
- EFX_BUG_ON_PARANOID(tx_queue->write_count != tx_queue->insert_count);
+ EFX_BUG_ON_PARANOID(tx_queue->write_count > tx_queue->insert_count);
rc = tso_start(&state, efx, skb);
if (rc)
netdev_tx_sent_queue(tx_queue->core_txq, skb->len);
- /* Pass off to hardware */
- efx_nic_push_buffers(tx_queue);
-
efx_tx_maybe_stop_queue(tx_queue);
+ /* Pass off to hardware */
+ if (!skb->xmit_more || netif_xmit_stopped(tx_queue->core_txq))
+ efx_nic_push_buffers(tx_queue);
+
tx_queue->tso_bursts++;
return NETDEV_TX_OK;
dma_unmap_single(&efx->pci_dev->dev, state.header_dma_addr,
state.header_unmap_len, DMA_TO_DEVICE);
- efx_enqueue_unwind(tx_queue);
+ efx_enqueue_unwind(tx_queue, old_insert_count);
return NETDEV_TX_OK;
}
*
* Copyright (C) 2003-2014 STMicroelectronics (R&D) Limited
* Author: Srinivas Kandagatla <srinivas.kandagatla@st.com>
- *
+ * Contributors: Giuseppe Cavallaro <peppe.cavallaro@st.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
#include <linux/of.h>
#include <linux/of_net.h>
+#define DWMAC_125MHZ 125000000
+#define DWMAC_50MHZ 50000000
+#define DWMAC_25MHZ 25000000
+#define DWMAC_2_5MHZ 2500000
+
+#define IS_PHY_IF_MODE_RGMII(iface) (iface == PHY_INTERFACE_MODE_RGMII || \
+ iface == PHY_INTERFACE_MODE_RGMII_ID || \
+ iface == PHY_INTERFACE_MODE_RGMII_RXID || \
+ iface == PHY_INTERFACE_MODE_RGMII_TXID)
+
+#define IS_PHY_IF_MODE_GBIT(iface) (IS_PHY_IF_MODE_RGMII(iface) || \
+ iface == PHY_INTERFACE_MODE_GMII)
+
+/* STiH4xx register definitions (STiH415/STiH416/STiH407/STiH410 families) */
+
/**
- * STi GMAC glue logic.
- * --------------------
- *
- * _
- * | \
- * --------|0 \ ETH_SEL_INTERNAL_NOTEXT_PHYCLK
- * phyclk | |___________________________________________
- * | | | (phyclk-in)
- * --------|1 / |
- * int-clk |_ / |
- * | _
- * | | \
- * |_______|1 \ ETH_SEL_TX_RETIME_CLK
- * | |___________________________
- * | | (tx-retime-clk)
- * _______|0 /
- * | |_ /
- * _ |
- * | \ |
- * --------|0 \ |
- * clk_125 | |__|
- * | | ETH_SEL_TXCLK_NOT_CLK125
- * --------|1 /
- * txclk |_ /
- *
- *
- * ETH_SEL_INTERNAL_NOTEXT_PHYCLK is valid only for RMII where PHY can
- * generate 50MHz clock or MAC can generate it.
- * This bit is configured by "st,ext-phyclk" property.
- *
- * ETH_SEL_TXCLK_NOT_CLK125 is only valid for gigabit modes, where the 125Mhz
- * clock either comes from clk-125 pin or txclk pin. This configuration is
- * totally driven by the board wiring. This bit is configured by
- * "st,tx-retime-src" property.
- *
- * TXCLK configuration is different for different phy interface modes
- * and changes according to link speed in modes like RGMII.
- *
* Below table summarizes the clock requirement and clock sources for
* supported phy interface modes with link speeds.
* ________________________________________________
* ------------------------------------------------
*| RGMII | 125Mhz | 25Mhz |
*| | clk-125/txclk | clkgen |
+ *| | clkgen | |
* ------------------------------------------------
*| RMII | n/a | 25Mhz |
*| | |clkgen/phyclk-in |
* ------------------------------------------------
*
- * TX lines are always retimed with a clk, which can vary depending
- * on the board configuration. Below is the table of these bits
- * in eth configuration register depending on source of retime clk.
- *
- *---------------------------------------------------------------
- * src | tx_rt_clk | int_not_ext_phyclk | txclk_n_clk125|
- *---------------------------------------------------------------
- * txclk | 0 | n/a | 1 |
- *---------------------------------------------------------------
- * ck_125| 0 | n/a | 0 |
- *---------------------------------------------------------------
- * phyclk| 1 | 0 | n/a |
- *---------------------------------------------------------------
- * clkgen| 1 | 1 | n/a |
- *---------------------------------------------------------------
+ * Register Configuration
+ *-------------------------------
+ * src |BIT(8)| BIT(7)| BIT(6)|
+ *-------------------------------
+ * txclk | 0 | n/a | 1 |
+ *-------------------------------
+ * ck_125| 0 | n/a | 0 |
+ *-------------------------------
+ * phyclk| 1 | 0 | n/a |
+ *-------------------------------
+ * clkgen| 1 | 1 | n/a |
+ *-------------------------------
*/
- /* Register definition */
+#define STIH4XX_RETIME_SRC_MASK GENMASK(8, 6)
+#define STIH4XX_ETH_SEL_TX_RETIME_CLK BIT(8)
+#define STIH4XX_ETH_SEL_INTERNAL_NOTEXT_PHYCLK BIT(7)
+#define STIH4XX_ETH_SEL_TXCLK_NOT_CLK125 BIT(6)
+
+/* STiD127 register definitions */
- /* 3 bits [8:6]
- * [6:6] ETH_SEL_TXCLK_NOT_CLK125
- * [7:7] ETH_SEL_INTERNAL_NOTEXT_PHYCLK
- * [8:8] ETH_SEL_TX_RETIME_CLK
- *
- */
+/**
+ *-----------------------
+ * src |BIT(6)| BIT(7)|
+ *-----------------------
+ * MII | 1 | n/a |
+ *-----------------------
+ * RMII | n/a | 1 |
+ * clkgen| | |
+ *-----------------------
+ * RMII | n/a | 0 |
+ * phyclk| | |
+ *-----------------------
+ * RGMII | 1 | n/a |
+ * clkgen| | |
+ *-----------------------
+ */
-#define TX_RETIME_SRC_MASK GENMASK(8, 6)
-#define ETH_SEL_TX_RETIME_CLK BIT(8)
-#define ETH_SEL_INTERNAL_NOTEXT_PHYCLK BIT(7)
-#define ETH_SEL_TXCLK_NOT_CLK125 BIT(6)
+#define STID127_RETIME_SRC_MASK GENMASK(7, 6)
+#define STID127_ETH_SEL_INTERNAL_NOTEXT_PHYCLK BIT(7)
+#define STID127_ETH_SEL_INTERNAL_NOTEXT_TXCLK BIT(6)
-#define ENMII_MASK GENMASK(5, 5)
-#define ENMII BIT(5)
+#define ENMII_MASK GENMASK(5, 5)
+#define ENMII BIT(5)
+#define EN_MASK GENMASK(1, 1)
+#define EN BIT(1)
/**
* 3 bits [4:2]
* 010-SGMII
* 100-RMII
*/
-#define MII_PHY_SEL_MASK GENMASK(4, 2)
-#define ETH_PHY_SEL_RMII BIT(4)
-#define ETH_PHY_SEL_SGMII BIT(3)
-#define ETH_PHY_SEL_RGMII BIT(2)
-#define ETH_PHY_SEL_GMII 0x0
-#define ETH_PHY_SEL_MII 0x0
-
-#define IS_PHY_IF_MODE_RGMII(iface) (iface == PHY_INTERFACE_MODE_RGMII || \
- iface == PHY_INTERFACE_MODE_RGMII_ID || \
- iface == PHY_INTERFACE_MODE_RGMII_RXID || \
- iface == PHY_INTERFACE_MODE_RGMII_TXID)
-
-#define IS_PHY_IF_MODE_GBIT(iface) (IS_PHY_IF_MODE_RGMII(iface) || \
- iface == PHY_INTERFACE_MODE_GMII)
+#define MII_PHY_SEL_MASK GENMASK(4, 2)
+#define ETH_PHY_SEL_RMII BIT(4)
+#define ETH_PHY_SEL_SGMII BIT(3)
+#define ETH_PHY_SEL_RGMII BIT(2)
+#define ETH_PHY_SEL_GMII 0x0
+#define ETH_PHY_SEL_MII 0x0
struct sti_dwmac {
- int interface;
- bool ext_phyclk;
- bool is_tx_retime_src_clk_125;
- struct clk *clk;
- int reg;
+ int interface; /* MII interface */
+ bool ext_phyclk; /* Clock from external PHY */
+ u32 tx_retime_src; /* TXCLK Retiming*/
+ struct clk *clk; /* PHY clock */
+ int ctrl_reg; /* GMAC glue-logic control register */
+ int clk_sel_reg; /* GMAC ext clk selection register */
struct device *dev;
struct regmap *regmap;
+ u32 speed;
};
static u32 phy_intf_sels[] = {
TX_RETIME_SRC_CLKGEN,
};
-static const char *const tx_retime_srcs[] = {
- [TX_RETIME_SRC_NA] = "",
- [TX_RETIME_SRC_TXCLK] = "txclk",
- [TX_RETIME_SRC_CLK_125] = "clk_125",
- [TX_RETIME_SRC_PHYCLK] = "phyclk",
- [TX_RETIME_SRC_CLKGEN] = "clkgen",
-};
-
-static u32 tx_retime_val[] = {
- [TX_RETIME_SRC_TXCLK] = ETH_SEL_TXCLK_NOT_CLK125,
+static u32 stih4xx_tx_retime_val[] = {
+ [TX_RETIME_SRC_TXCLK] = STIH4XX_ETH_SEL_TXCLK_NOT_CLK125,
[TX_RETIME_SRC_CLK_125] = 0x0,
- [TX_RETIME_SRC_PHYCLK] = ETH_SEL_TX_RETIME_CLK,
- [TX_RETIME_SRC_CLKGEN] = ETH_SEL_TX_RETIME_CLK |
- ETH_SEL_INTERNAL_NOTEXT_PHYCLK,
+ [TX_RETIME_SRC_PHYCLK] = STIH4XX_ETH_SEL_TX_RETIME_CLK,
+ [TX_RETIME_SRC_CLKGEN] = STIH4XX_ETH_SEL_TX_RETIME_CLK
+ | STIH4XX_ETH_SEL_INTERNAL_NOTEXT_PHYCLK,
};
-static void setup_retime_src(struct sti_dwmac *dwmac, u32 spd)
+static void stih4xx_fix_retime_src(void *priv, u32 spd)
{
- u32 src = 0, freq = 0;
-
- if (spd == SPEED_100) {
- if (dwmac->interface == PHY_INTERFACE_MODE_MII ||
- dwmac->interface == PHY_INTERFACE_MODE_GMII) {
- src = TX_RETIME_SRC_TXCLK;
- } else if (dwmac->interface == PHY_INTERFACE_MODE_RMII) {
- if (dwmac->ext_phyclk) {
- src = TX_RETIME_SRC_PHYCLK;
- } else {
- src = TX_RETIME_SRC_CLKGEN;
- freq = 50000000;
- }
-
- } else if (IS_PHY_IF_MODE_RGMII(dwmac->interface)) {
+ struct sti_dwmac *dwmac = priv;
+ u32 src = dwmac->tx_retime_src;
+ u32 reg = dwmac->ctrl_reg;
+ u32 freq = 0;
+
+ if (dwmac->interface == PHY_INTERFACE_MODE_MII) {
+ src = TX_RETIME_SRC_TXCLK;
+ } else if (dwmac->interface == PHY_INTERFACE_MODE_RMII) {
+ if (dwmac->ext_phyclk) {
+ src = TX_RETIME_SRC_PHYCLK;
+ } else {
src = TX_RETIME_SRC_CLKGEN;
- freq = 25000000;
+ freq = DWMAC_50MHZ;
}
+ } else if (IS_PHY_IF_MODE_RGMII(dwmac->interface)) {
+ /* On GiGa clk source can be either ext or from clkgen */
+ if (spd == SPEED_1000) {
+ freq = DWMAC_125MHZ;
+ } else {
+ /* Switch to clkgen for these speeds */
+ src = TX_RETIME_SRC_CLKGEN;
+ if (spd == SPEED_100)
+ freq = DWMAC_25MHZ;
+ else if (spd == SPEED_10)
+ freq = DWMAC_2_5MHZ;
+ }
+ }
- if (src == TX_RETIME_SRC_CLKGEN && dwmac->clk)
- clk_set_rate(dwmac->clk, freq);
+ if (src == TX_RETIME_SRC_CLKGEN && dwmac->clk && freq)
+ clk_set_rate(dwmac->clk, freq);
- } else if (spd == SPEED_1000) {
- if (dwmac->is_tx_retime_src_clk_125)
- src = TX_RETIME_SRC_CLK_125;
- else
- src = TX_RETIME_SRC_TXCLK;
+ regmap_update_bits(dwmac->regmap, reg, STIH4XX_RETIME_SRC_MASK,
+ stih4xx_tx_retime_val[src]);
+}
+
+static void stid127_fix_retime_src(void *priv, u32 spd)
+{
+ struct sti_dwmac *dwmac = priv;
+ u32 reg = dwmac->ctrl_reg;
+ u32 freq = 0;
+ u32 val = 0;
+
+ if (dwmac->interface == PHY_INTERFACE_MODE_MII) {
+ val = STID127_ETH_SEL_INTERNAL_NOTEXT_TXCLK;
+ } else if (dwmac->interface == PHY_INTERFACE_MODE_RMII) {
+ if (!dwmac->ext_phyclk) {
+ val = STID127_ETH_SEL_INTERNAL_NOTEXT_PHYCLK;
+ freq = DWMAC_50MHZ;
+ }
+ } else if (IS_PHY_IF_MODE_RGMII(dwmac->interface)) {
+ val = STID127_ETH_SEL_INTERNAL_NOTEXT_TXCLK;
+ if (spd == SPEED_1000)
+ freq = DWMAC_125MHZ;
+ else if (spd == SPEED_100)
+ freq = DWMAC_25MHZ;
+ else if (spd == SPEED_10)
+ freq = DWMAC_2_5MHZ;
}
- regmap_update_bits(dwmac->regmap, dwmac->reg,
- TX_RETIME_SRC_MASK, tx_retime_val[src]);
+ if (dwmac->clk && freq)
+ clk_set_rate(dwmac->clk, freq);
+
+ regmap_update_bits(dwmac->regmap, reg, STID127_RETIME_SRC_MASK, val);
}
-static void sti_dwmac_exit(struct platform_device *pdev, void *priv)
+static void sti_dwmac_ctrl_init(struct sti_dwmac *dwmac)
{
- struct sti_dwmac *dwmac = priv;
+ struct regmap *regmap = dwmac->regmap;
+ int iface = dwmac->interface;
+ struct device *dev = dwmac->dev;
+ struct device_node *np = dev->of_node;
+ u32 reg = dwmac->ctrl_reg;
+ u32 val;
if (dwmac->clk)
- clk_disable_unprepare(dwmac->clk);
+ clk_prepare_enable(dwmac->clk);
+
+ if (of_property_read_bool(np, "st,gmac_en"))
+ regmap_update_bits(regmap, reg, EN_MASK, EN);
+
+ regmap_update_bits(regmap, reg, MII_PHY_SEL_MASK, phy_intf_sels[iface]);
+
+ val = (iface == PHY_INTERFACE_MODE_REVMII) ? 0 : ENMII;
+ regmap_update_bits(regmap, reg, ENMII_MASK, val);
+}
+
+static int stix4xx_init(struct platform_device *pdev, void *priv)
+{
+ struct sti_dwmac *dwmac = priv;
+ u32 spd = dwmac->speed;
+
+ sti_dwmac_ctrl_init(dwmac);
+
+ stih4xx_fix_retime_src(priv, spd);
+
+ return 0;
}
-static void sti_fix_mac_speed(void *priv, unsigned int spd)
+static int stid127_init(struct platform_device *pdev, void *priv)
{
struct sti_dwmac *dwmac = priv;
+ u32 spd = dwmac->speed;
- setup_retime_src(dwmac, spd);
+ sti_dwmac_ctrl_init(dwmac);
- return;
+ stid127_fix_retime_src(priv, spd);
+
+ return 0;
}
+static void sti_dwmac_exit(struct platform_device *pdev, void *priv)
+{
+ struct sti_dwmac *dwmac = priv;
+
+ if (dwmac->clk)
+ clk_disable_unprepare(dwmac->clk);
+}
static int sti_dwmac_parse_data(struct sti_dwmac *dwmac,
struct platform_device *pdev)
{
res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "sti-ethconf");
if (!res)
return -ENODATA;
+ dwmac->ctrl_reg = res->start;
+
+ /* clk selection from extra syscfg register */
+ dwmac->clk_sel_reg = -ENXIO;
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "sti-clkconf");
+ if (res)
+ dwmac->clk_sel_reg = res->start;
regmap = syscon_regmap_lookup_by_phandle(np, "st,syscon");
if (IS_ERR(regmap))
dwmac->dev = dev;
dwmac->interface = of_get_phy_mode(np);
dwmac->regmap = regmap;
- dwmac->reg = res->start;
dwmac->ext_phyclk = of_property_read_bool(np, "st,ext-phyclk");
- dwmac->is_tx_retime_src_clk_125 = false;
+ dwmac->tx_retime_src = TX_RETIME_SRC_NA;
+ dwmac->speed = SPEED_100;
if (IS_PHY_IF_MODE_GBIT(dwmac->interface)) {
const char *rs;
+ dwmac->tx_retime_src = TX_RETIME_SRC_CLKGEN;
err = of_property_read_string(np, "st,tx-retime-src", &rs);
- if (err < 0) {
- dev_err(dev, "st,tx-retime-src not specified\n");
- return err;
- }
+ if (err < 0)
+ dev_warn(dev, "Use internal clock source\n");
if (!strcasecmp(rs, "clk_125"))
- dwmac->is_tx_retime_src_clk_125 = true;
+ dwmac->tx_retime_src = TX_RETIME_SRC_CLK_125;
+ else if (!strcasecmp(rs, "txclk"))
+ dwmac->tx_retime_src = TX_RETIME_SRC_TXCLK;
+
+ dwmac->speed = SPEED_1000;
}
dwmac->clk = devm_clk_get(dev, "sti-ethclk");
-
- if (IS_ERR(dwmac->clk))
+ if (IS_ERR(dwmac->clk)) {
+ dev_warn(dev, "No phy clock provided...\n");
dwmac->clk = NULL;
-
- return 0;
-}
-
-static int sti_dwmac_init(struct platform_device *pdev, void *priv)
-{
- struct sti_dwmac *dwmac = priv;
- struct regmap *regmap = dwmac->regmap;
- int iface = dwmac->interface;
- u32 reg = dwmac->reg;
- u32 val, spd;
-
- if (dwmac->clk)
- clk_prepare_enable(dwmac->clk);
-
- regmap_update_bits(regmap, reg, MII_PHY_SEL_MASK, phy_intf_sels[iface]);
-
- val = (iface == PHY_INTERFACE_MODE_REVMII) ? 0 : ENMII;
- regmap_update_bits(regmap, reg, ENMII_MASK, val);
-
- if (IS_PHY_IF_MODE_GBIT(iface))
- spd = SPEED_1000;
- else
- spd = SPEED_100;
-
- setup_retime_src(dwmac, spd);
+ }
return 0;
}
return dwmac;
}
-const struct stmmac_of_data sti_gmac_data = {
- .fix_mac_speed = sti_fix_mac_speed,
+const struct stmmac_of_data stih4xx_dwmac_data = {
+ .fix_mac_speed = stih4xx_fix_retime_src,
+ .setup = sti_dwmac_setup,
+ .init = stix4xx_init,
+ .exit = sti_dwmac_exit,
+};
+
+const struct stmmac_of_data stid127_dwmac_data = {
+ .fix_mac_speed = stid127_fix_retime_src,
.setup = sti_dwmac_setup,
- .init = sti_dwmac_init,
+ .init = stid127_init,
.exit = sti_dwmac_exit,
};
extern const struct stmmac_of_data sun7i_gmac_data;
#endif
#ifdef CONFIG_DWMAC_STI
-extern const struct stmmac_of_data sti_gmac_data;
+extern const struct stmmac_of_data stih4xx_dwmac_data;
+extern const struct stmmac_of_data stid127_dwmac_data;
#endif
#ifdef CONFIG_DWMAC_SOCFPGA
extern const struct stmmac_of_data socfpga_gmac_data;
{ .compatible = "allwinner,sun7i-a20-gmac", .data = &sun7i_gmac_data},
#endif
#ifdef CONFIG_DWMAC_STI
- { .compatible = "st,stih415-dwmac", .data = &sti_gmac_data},
- { .compatible = "st,stih416-dwmac", .data = &sti_gmac_data},
- { .compatible = "st,stid127-dwmac", .data = &sti_gmac_data},
+ { .compatible = "st,stih415-dwmac", .data = &stih4xx_dwmac_data},
+ { .compatible = "st,stih416-dwmac", .data = &stih4xx_dwmac_data},
+ { .compatible = "st,stid127-dwmac", .data = &stid127_dwmac_data},
+ { .compatible = "st,stih407-dwmac", .data = &stih4xx_dwmac_data},
#endif
#ifdef CONFIG_DWMAC_SOCFPGA
{ .compatible = "altr,socfpga-stmmac", .data = &socfpga_gmac_data },
if (of_property_read_u32(np, "snps,phy-addr", &plat->phy_addr) == 0)
dev_warn(&pdev->dev, "snps,phy-addr property is deprecated\n");
- plat->mdio_bus_data = devm_kzalloc(&pdev->dev,
- sizeof(struct stmmac_mdio_bus_data),
- GFP_KERNEL);
+ if (plat->phy_bus_name)
+ plat->mdio_bus_data = NULL;
+ else
+ plat->mdio_bus_data =
+ devm_kzalloc(&pdev->dev,
+ sizeof(struct stmmac_mdio_bus_data),
+ GFP_KERNEL);
- plat->force_sf_dma_mode = of_property_read_bool(np, "snps,force_sf_dma_mode");
+ plat->force_sf_dma_mode =
+ of_property_read_bool(np, "snps,force_sf_dma_mode");
/* Set the maxmtu to a default of JUMBO_LEN in case the
* parameter is not present in the device tree.
return ret;
}
+static int cpsw_remove_child_device(struct device *dev, void *c)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+
+ of_device_unregister(pdev);
+
+ return 0;
+}
+
static int cpsw_remove(struct platform_device *pdev)
{
struct net_device *ndev = platform_get_drvdata(pdev);
cpdma_chan_destroy(priv->rxch);
cpdma_ctlr_destroy(priv->dma);
pm_runtime_disable(&pdev->dev);
+ device_for_each_child(&pdev->dev, NULL, cpsw_remove_child_device);
if (priv->data.dual_emac)
free_netdev(cpsw_get_slave_ndev(priv, 1));
free_netdev(ndev);
static void cpdma_desc_pool_destroy(struct cpdma_desc_pool *pool)
{
- unsigned long flags;
-
if (!pool)
return;
- spin_lock_irqsave(&pool->lock, flags);
WARN_ON(pool->used_desc);
if (pool->cpumap) {
dma_free_coherent(pool->dev, pool->mem_size, pool->cpumap,
} else {
iounmap(pool->iomap);
}
- spin_unlock_irqrestore(&pool->lock, flags);
}
static inline dma_addr_t desc_phys(struct cpdma_desc_pool *pool,
cpdma_chan_stop(chan);
ctlr->channels[chan->chan_num] = NULL;
spin_unlock_irqrestore(&ctlr->lock, flags);
- kfree(chan);
return 0;
}
EXPORT_SYMBOL_GPL(cpdma_chan_destroy);
* data: network byte order
* return: host byte order
*/
-static u32 comp_hash(u8 *key, int klen, u8 *data, int dlen)
+static u32 comp_hash(u8 *key, int klen, void *data, int dlen)
{
union sub_key subk;
int k_next = 4;
for (i = 0; i < dlen; i++) {
subk.kb = key[k_next];
k_next = (k_next + 1) % klen;
- dt = data[i];
+ dt = ((u8 *)data)[i];
for (j = 0; j < 8; j++) {
if (dt & 0x80)
ret ^= subk.ka;
static bool netvsc_set_hash(u32 *hash, struct sk_buff *skb)
{
- struct iphdr *iphdr;
+ struct flow_keys flow;
int data_len;
- bool ret = false;
- if (eth_hdr(skb)->h_proto != htons(ETH_P_IP))
+ if (!skb_flow_dissect(skb, &flow) || flow.n_proto != htons(ETH_P_IP))
return false;
- iphdr = ip_hdr(skb);
+ if (flow.ip_proto == IPPROTO_TCP)
+ data_len = 12;
+ else
+ data_len = 8;
- if (iphdr->version == 4) {
- if (iphdr->protocol == IPPROTO_TCP)
- data_len = 12;
- else
- data_len = 8;
- *hash = comp_hash(netvsc_hash_key, HASH_KEYLEN,
- (u8 *)&iphdr->saddr, data_len);
- ret = true;
- }
+ *hash = comp_hash(netvsc_hash_key, HASH_KEYLEN, &flow, data_len);
- return ret;
+ return true;
}
static u16 netvsc_select_queue(struct net_device *ndev, struct sk_buff *skb,
*/
if (q->flags & IFF_VNET_HDR)
features |= vlan->tap_features;
- if (netif_needs_gso(skb, features)) {
+ if (netif_needs_gso(dev, skb, features)) {
struct sk_buff *segs = __skb_gso_segment(skb, features, false);
if (IS_ERR(segs))
if (rc)
dev_err(&phydev->dev, "failed to set led mode\n");
- phy_write(phydev, MII_KSZPHY_OMSO, val);
rc = ksz_config_flags(phydev);
+ if (rc < 0)
+ return rc;
+ rc = phy_write(phydev, MII_KSZPHY_OMSO, val);
return rc < 0 ? rc : 0;
}
static int rtl8152_suspend(struct usb_interface *intf, pm_message_t message)
{
struct r8152 *tp = usb_get_intfdata(intf);
+ struct net_device *netdev = tp->netdev;
+ int ret = 0;
mutex_lock(&tp->control);
- if (PMSG_IS_AUTO(message))
+ if (PMSG_IS_AUTO(message)) {
+ if (netif_running(netdev) && work_busy(&tp->schedule.work)) {
+ ret = -EBUSY;
+ goto out1;
+ }
+
set_bit(SELECTIVE_SUSPEND, &tp->flags);
- else
- netif_device_detach(tp->netdev);
+ } else {
+ netif_device_detach(netdev);
+ }
- if (netif_running(tp->netdev)) {
+ if (netif_running(netdev)) {
clear_bit(WORK_ENABLE, &tp->flags);
usb_kill_urb(tp->intr_urb);
- cancel_delayed_work_sync(&tp->schedule);
tasklet_disable(&tp->tl);
if (test_bit(SELECTIVE_SUSPEND, &tp->flags)) {
rtl_stop_rx(tp);
rtl_runtime_suspend_enable(tp, true);
} else {
+ cancel_delayed_work_sync(&tp->schedule);
tp->rtl_ops.down(tp);
}
tasklet_enable(&tp->tl);
}
-
+out1:
mutex_unlock(&tp->control);
- return 0;
+ return ret;
}
static int rtl8152_resume(struct usb_interface *intf)
/* Host can handle any s/g split between our header and packet data */
bool any_header_sg;
- /* enable config space updates */
- bool config_enable;
-
/* Active statistics */
struct virtnet_stats __percpu *stats;
/* Work struct for config space updates */
struct work_struct config_work;
- /* Lock for config space updates */
- struct mutex config_lock;
-
/* Does the affinity hint is set for virtqueues? */
bool affinity_hint_set;
int qnum = skb_get_queue_mapping(skb);
struct send_queue *sq = &vi->sq[qnum];
int err;
+ struct netdev_queue *txq = netdev_get_tx_queue(dev, qnum);
+ bool kick = !skb->xmit_more;
/* Free up any pending old buffers before queueing new ones. */
free_old_xmit_skbs(sq);
}
}
- if (__netif_subqueue_stopped(dev, qnum) || !skb->xmit_more)
+ if (kick || netif_xmit_stopped(txq))
virtqueue_kick(sq->vq);
return NETDEV_TX_OK;
container_of(work, struct virtnet_info, config_work);
u16 v;
- mutex_lock(&vi->config_lock);
- if (!vi->config_enable)
- goto done;
-
if (virtio_cread_feature(vi->vdev, VIRTIO_NET_F_STATUS,
struct virtio_net_config, status, &v) < 0)
- goto done;
+ return;
if (v & VIRTIO_NET_S_ANNOUNCE) {
netdev_notify_peers(vi->dev);
v &= VIRTIO_NET_S_LINK_UP;
if (vi->status == v)
- goto done;
+ return;
vi->status = v;
netif_carrier_off(vi->dev);
netif_tx_stop_all_queues(vi->dev);
}
-done:
- mutex_unlock(&vi->config_lock);
}
static void virtnet_config_changed(struct virtio_device *vdev)
u64_stats_init(&virtnet_stats->rx_syncp);
}
- mutex_init(&vi->config_lock);
- vi->config_enable = true;
INIT_WORK(&vi->config_work, virtnet_config_changed_work);
/* If we can receive ANY GSO packets, we must allocate large ones. */
goto free_vqs;
}
+ virtio_device_ready(vdev);
+
/* Last of all, set up some receive buffers. */
for (i = 0; i < vi->curr_queue_pairs; i++) {
try_fill_recv(&vi->rq[i], GFP_KERNEL);
return 0;
free_recv_bufs:
+ vi->vdev->config->reset(vdev);
+
free_receive_bufs(vi);
unregister_netdev(dev);
free_vqs:
unregister_hotcpu_notifier(&vi->nb);
- /* Prevent config work handler from accessing the device. */
- mutex_lock(&vi->config_lock);
- vi->config_enable = false;
- mutex_unlock(&vi->config_lock);
+ /* Make sure no work handler is accessing the device. */
+ flush_work(&vi->config_work);
unregister_netdev(vi->dev);
remove_vq_common(vi);
- flush_work(&vi->config_work);
-
free_percpu(vi->stats);
free_netdev(vi->dev);
}
unregister_hotcpu_notifier(&vi->nb);
- /* Prevent config work handler from accessing the device */
- mutex_lock(&vi->config_lock);
- vi->config_enable = false;
- mutex_unlock(&vi->config_lock);
+ /* Make sure no work handler is accessing the device */
+ flush_work(&vi->config_work);
netif_device_detach(vi->dev);
cancel_delayed_work_sync(&vi->refill);
remove_vq_common(vi);
- flush_work(&vi->config_work);
-
return 0;
}
if (err)
return err;
+ virtio_device_ready(vdev);
+
if (netif_running(vi->dev)) {
for (i = 0; i < vi->curr_queue_pairs; i++)
if (!try_fill_recv(&vi->rq[i], GFP_KERNEL))
netif_device_attach(vi->dev);
- mutex_lock(&vi->config_lock);
- vi->config_enable = true;
- mutex_unlock(&vi->config_lock);
-
rtnl_lock();
virtnet_set_queues(vi, vi->curr_queue_pairs);
rtnl_unlock();
if (!in6_dev)
goto out;
- if (!pskb_may_pull(skb, skb->len))
- goto out;
-
iphdr = ipv6_hdr(skb);
saddr = &iphdr->saddr;
daddr = &iphdr->daddr;
struct pcpu_sw_netstats *tx_stats, *rx_stats;
union vxlan_addr loopback;
union vxlan_addr *remote_ip = &dst_vxlan->default_dst.remote_ip;
+ struct net_device *dev = skb->dev;
+ int len = skb->len;
tx_stats = this_cpu_ptr(src_vxlan->dev->tstats);
rx_stats = this_cpu_ptr(dst_vxlan->dev->tstats);
u64_stats_update_begin(&tx_stats->syncp);
tx_stats->tx_packets++;
- tx_stats->tx_bytes += skb->len;
+ tx_stats->tx_bytes += len;
u64_stats_update_end(&tx_stats->syncp);
if (netif_rx(skb) == NET_RX_SUCCESS) {
u64_stats_update_begin(&rx_stats->syncp);
rx_stats->rx_packets++;
- rx_stats->rx_bytes += skb->len;
+ rx_stats->rx_bytes += len;
u64_stats_update_end(&rx_stats->syncp);
} else {
- skb->dev->stats.rx_dropped++;
+ dev->stats.rx_dropped++;
}
}
return arp_reduce(dev, skb);
#if IS_ENABLED(CONFIG_IPV6)
else if (ntohs(eth->h_proto) == ETH_P_IPV6 &&
- skb->len >= sizeof(struct ipv6hdr) + sizeof(struct nd_msg) &&
+ pskb_may_pull(skb, sizeof(struct ipv6hdr)
+ + sizeof(struct nd_msg)) &&
ipv6_hdr(skb)->nexthdr == IPPROTO_ICMPV6) {
struct nd_msg *msg;
msg->icmph.icmp6_type == NDISC_NEIGHBOUR_SOLICITATION)
return neigh_reduce(dev, skb);
}
+ eth = eth_hdr(skb);
#endif
}
if (unlikely(!netif_carrier_ok(dev) ||
(slots > 1 && !xennet_can_sg(dev)) ||
- netif_needs_gso(skb, netif_skb_features(skb)))) {
+ netif_needs_gso(dev, skb, netif_skb_features(skb)))) {
spin_unlock_irqrestore(&queue->tx_lock, flags);
goto drop;
}
blk_queue_max_hw_sectors(rq, nr_max_blk << 3); /* 8 * 512 = blk_size */
blk_queue_max_segments(rq, nr_max_blk);
queue_flag_set_unlocked(QUEUE_FLAG_NONROT, rq);
+ queue_flag_clear_unlocked(QUEUE_FLAG_ADD_RANDOM, rq);
scm_blk_dev_cluster_setup(bdev);
bdev->gendisk = alloc_disk(SCM_NR_PARTS);
goto out;
}
queue_flag_set_unlocked(QUEUE_FLAG_NONROT, xpram_queues[i]);
+ queue_flag_clear_unlocked(QUEUE_FLAG_ADD_RANDOM, xpram_queues[i]);
blk_queue_make_request(xpram_queues[i], xpram_make_request);
blk_queue_logical_block_size(xpram_queues[i], 4096);
}
switch (param) {
case VIRTIO_PARAM_CONFIG_CHANGED:
- {
- struct virtio_driver *drv;
- drv = container_of(vq->vdev->dev.driver,
- struct virtio_driver, driver);
- if (drv->config_changed)
- drv->config_changed(vq->vdev);
-
+ virtio_config_changed(vq->vdev);
break;
- }
case VIRTIO_PARAM_DEV_ADD:
schedule_work(&hotplug_work);
break;
vring_interrupt(0, vq);
}
if (test_bit(0, &vcdev->indicators2)) {
- drv = container_of(vcdev->vdev.dev.driver,
- struct virtio_driver, driver);
-
- if (drv && drv->config_changed)
- drv->config_changed(&vcdev->vdev);
+ virtio_config_changed(&vcdev->vdev);
clear_bit(0, &vcdev->indicators2);
}
}
config BLK_DEV_SD
tristate "SCSI disk support"
depends on SCSI
- select CRC_T10DIF if BLK_DEV_INTEGRITY
---help---
If you want to use SCSI hard disks, Fibre Channel disks,
Serial ATA (SATA) or Parallel ATA (PATA) hard disks,
cxgb4_l2t_send(csk->cdev->ports[csk->port_id], skb, csk->l2t);
}
+#if IS_ENABLED(CONFIG_IPV6)
static void send_act_open_req6(struct cxgbi_sock *csk, struct sk_buff *skb,
struct l2t_entry *e)
{
cxgb4_l2t_send(csk->cdev->ports[csk->port_id], skb, csk->l2t);
}
+#endif
static void send_close_req(struct cxgbi_sock *csk)
{
static void csk_act_open_retry_timer(unsigned long data)
{
- struct sk_buff *skb;
+ struct sk_buff *skb = NULL;
struct cxgbi_sock *csk = (struct cxgbi_sock *)data;
struct cxgb4_lld_info *lldi = cxgbi_cdev_priv(csk->cdev);
void (*send_act_open_func)(struct cxgbi_sock *, struct sk_buff *,
if (csk->csk_family == AF_INET) {
send_act_open_func = send_act_open_req;
skb = alloc_wr(size, 0, GFP_ATOMIC);
+#if IS_ENABLED(CONFIG_IPV6)
} else {
send_act_open_func = send_act_open_req6;
skb = alloc_wr(size6, 0, GFP_ATOMIC);
+#endif
}
if (!skb)
cxgbi_sock_set_flag(csk, CTPF_HAS_ATID);
cxgbi_sock_get(csk);
- n = dst_neigh_lookup(csk->dst, &csk->daddr.sin_addr.s_addr);
- if (!n) {
- pr_err("%s, can't get neighbour of csk->dst.\n", ndev->name);
- goto rel_resource;
- }
csk->l2t = cxgb4_l2t_get(lldi->l2t, n, ndev, 0);
if (!csk->l2t) {
pr_err("%s, cannot alloc l2t.\n", ndev->name);
if (csk->csk_family == AF_INET)
skb = alloc_wr(size, 0, GFP_NOIO);
+#if IS_ENABLED(CONFIG_IPV6)
else
skb = alloc_wr(size6, 0, GFP_NOIO);
+#endif
if (!skb)
goto rel_resource;
cxgbi_sock_set_state(csk, CTP_ACTIVE_OPEN);
if (csk->csk_family == AF_INET)
send_act_open_req(csk, skb, csk->l2t);
+#if IS_ENABLED(CONFIG_IPV6)
else
send_act_open_req6(csk, skb, csk->l2t);
+#endif
neigh_release(n);
return 0;
return 0;
}
-#if IS_ENABLED(CONFIG_IPV6)
-static int cxgbi_inet6addr_handler(struct notifier_block *this,
- unsigned long event, void *data)
-{
- struct inet6_ifaddr *ifa = data;
- struct net_device *event_dev = ifa->idev->dev;
- struct cxgbi_device *cdev;
- int ret = NOTIFY_DONE;
-
- if (event_dev->priv_flags & IFF_802_1Q_VLAN)
- event_dev = vlan_dev_real_dev(event_dev);
-
- cdev = cxgbi_device_find_by_netdev_rcu(event_dev, NULL);
-
- if (!cdev)
- return ret;
-
- switch (event) {
- case NETDEV_UP:
- ret = cxgb4_clip_get(event_dev,
- (const struct in6_addr *)
- ((ifa)->addr.s6_addr));
- if (ret < 0)
- return ret;
-
- ret = NOTIFY_OK;
- break;
-
- case NETDEV_DOWN:
- cxgb4_clip_release(event_dev,
- (const struct in6_addr *)
- ((ifa)->addr.s6_addr));
- ret = NOTIFY_OK;
- break;
-
- default:
- break;
- }
-
- return ret;
-}
-
-static struct notifier_block cxgbi_inet6addr_notifier = {
- .notifier_call = cxgbi_inet6addr_handler
-};
-
-/* Retrieve IPv6 addresses from a root device (bond, vlan) associated with
- * a physical device.
- * The physical device reference is needed to send the actual CLIP command.
- */
-static int update_dev_clip(struct net_device *root_dev, struct net_device *dev)
-{
- struct inet6_dev *idev = NULL;
- struct inet6_ifaddr *ifa;
- int ret = 0;
-
- idev = __in6_dev_get(root_dev);
- if (!idev)
- return ret;
-
- read_lock_bh(&idev->lock);
- list_for_each_entry(ifa, &idev->addr_list, if_list) {
- pr_info("updating the clip for addr %pI6\n",
- ifa->addr.s6_addr);
- ret = cxgb4_clip_get(dev, (const struct in6_addr *)
- ifa->addr.s6_addr);
- if (ret < 0)
- break;
- }
-
- read_unlock_bh(&idev->lock);
- return ret;
-}
-
-static int update_root_dev_clip(struct net_device *dev)
-{
- struct net_device *root_dev = NULL;
- int i, ret = 0;
-
- /* First populate the real net device's IPv6 address */
- ret = update_dev_clip(dev, dev);
- if (ret)
- return ret;
-
- /* Parse all bond and vlan devices layered on top of the physical dev */
- root_dev = netdev_master_upper_dev_get(dev);
- if (root_dev) {
- ret = update_dev_clip(root_dev, dev);
- if (ret)
- return ret;
- }
-
- for (i = 0; i < VLAN_N_VID; i++) {
- root_dev = __vlan_find_dev_deep_rcu(dev, htons(ETH_P_8021Q), i);
- if (!root_dev)
- continue;
-
- ret = update_dev_clip(root_dev, dev);
- if (ret)
- break;
- }
- return ret;
-}
-
-static void cxgbi_update_clip(struct cxgbi_device *cdev)
-{
- int i;
-
- rcu_read_lock();
-
- for (i = 0; i < cdev->nports; i++) {
- struct net_device *dev = cdev->ports[i];
- int ret = 0;
-
- if (dev)
- ret = update_root_dev_clip(dev);
- if (ret < 0)
- break;
- }
- rcu_read_unlock();
-}
-#endif /* IS_ENABLED(CONFIG_IPV6) */
-
static void *t4_uld_add(const struct cxgb4_lld_info *lldi)
{
struct cxgbi_device *cdev;
switch (state) {
case CXGB4_STATE_UP:
pr_info("cdev 0x%p, UP.\n", cdev);
-#if IS_ENABLED(CONFIG_IPV6)
- cxgbi_update_clip(cdev);
-#endif
- /* re-initialize */
break;
case CXGB4_STATE_START_RECOVERY:
pr_info("cdev 0x%p, RECOVERY.\n", cdev);
return rc;
cxgb4_register_uld(CXGB4_ULD_ISCSI, &cxgb4i_uld_info);
-#if IS_ENABLED(CONFIG_IPV6)
- register_inet6addr_notifier(&cxgbi_inet6addr_notifier);
-#endif
return 0;
}
static void __exit cxgb4i_exit_module(void)
{
-#if IS_ENABLED(CONFIG_IPV6)
- unregister_inet6addr_notifier(&cxgbi_inet6addr_notifier);
-#endif
cxgb4_unregister_uld(CXGB4_ULD_ISCSI);
cxgbi_device_unregister_all(CXGBI_FLAG_DEV_T4);
cxgbi_iscsi_cleanup(&cxgb4i_iscsi_transport, &cxgb4i_stt);
}
EXPORT_SYMBOL_GPL(cxgbi_device_find_by_netdev_rcu);
+#if IS_ENABLED(CONFIG_IPV6)
static struct cxgbi_device *cxgbi_device_find_by_mac(struct net_device *ndev,
int *port)
{
ndev, ndev->name);
return NULL;
}
+#endif
void cxgbi_hbas_remove(struct cxgbi_device *cdev)
{
rq = blk_get_request(q, rw, GFP_NOIO);
- if (!rq) {
+ if (IS_ERR(rq)) {
sdev_printk(KERN_INFO, sdev,
"%s: blk_get_request failed\n", __func__);
return NULL;
rq = blk_get_request(sdev->request_queue,
(cmd != INQUIRY) ? WRITE : READ, GFP_NOIO);
- if (!rq) {
+ if (IS_ERR(rq)) {
sdev_printk(KERN_INFO, sdev, "get_req: blk_get_request failed");
return NULL;
}
retry:
req = blk_get_request(sdev->request_queue, WRITE, GFP_NOIO);
- if (!req)
+ if (IS_ERR(req))
return SCSI_DH_RES_TEMP_UNAVAIL;
blk_rq_set_block_pc(req);
struct request *req;
req = blk_get_request(h->sdev->request_queue, WRITE, GFP_ATOMIC);
- if (!req)
+ if (IS_ERR(req))
return SCSI_DH_RES_TEMP_UNAVAIL;
blk_rq_set_block_pc(req);
rq = blk_get_request(q, rw, GFP_NOIO);
- if (!rq) {
+ if (IS_ERR(rq)) {
sdev_printk(KERN_INFO, sdev,
"get_rdac_req: blk_get_request failed.\n");
return NULL;
struct request *req;
req = blk_get_request(q, has_write ? WRITE : READ, flags);
- if (unlikely(!req))
- return ERR_PTR(-ENOMEM);
+ if (IS_ERR(req))
+ return req;
blk_rq_set_block_pc(req);
return req;
int write = (data_direction == DMA_TO_DEVICE);
req = blk_get_request(SRpnt->stp->device->request_queue, write, GFP_KERNEL);
- if (!req)
+ if (IS_ERR(req))
return DRIVER_ERROR << 24;
blk_rq_set_block_pc(req);
* request becomes available
*/
req = blk_get_request(sdev->request_queue, READ, GFP_KERNEL);
+ if (IS_ERR(req))
+ return;
blk_rq_set_block_pc(req);
int ret = DRIVER_ERROR << 24;
req = blk_get_request(sdev->request_queue, write, __GFP_WAIT);
- if (!req)
+ if (IS_ERR(req))
return ret;
blk_rq_set_block_pc(req);
if (req->mq_ctx) {
/*
- * In the MQ case the command gets freed by __blk_mq_end_io,
+ * In the MQ case the command gets freed by __blk_mq_end_request,
* so we have to do all cleanup that depends on it earlier.
*
* We also can't kick the queues from irq context, so we
*/
scsi_mq_uninit_cmd(cmd);
- __blk_mq_end_io(req, error);
+ __blk_mq_end_request(req, error);
if (scsi_target(sdev)->single_lun ||
!list_empty(&sdev->host->starved_list))
next_rq->special = bidi_sdb;
}
+ blk_mq_start_request(req);
+
return scsi_setup_cmnd(sdev, req);
}
blk_mq_complete_request(cmd->request);
}
-static int scsi_queue_rq(struct blk_mq_hw_ctx *hctx, struct request *req)
+static int scsi_queue_rq(struct blk_mq_hw_ctx *hctx, struct request *req,
+ bool last)
{
struct request_queue *q = req->q;
struct scsi_device *sdev = q->queuedata;
if (!scsi_host_queue_ready(q, shost, sdev))
goto out_dec_target_busy;
+
if (!(req->cmd_flags & REQ_DONTPREP)) {
ret = prep_to_mq(scsi_mq_prep_fn(req));
if (ret)
goto out_dec_host_busy;
req->cmd_flags |= REQ_DONTPREP;
+ } else {
+ blk_mq_start_request(req);
}
scsi_init_cmd_errh(cmd);
return ret;
}
+static enum blk_eh_timer_return scsi_timeout(struct request *req,
+ bool reserved)
+{
+ if (reserved)
+ return BLK_EH_RESET_TIMER;
+ return scsi_times_out(req);
+}
+
static int scsi_init_request(void *data, struct request *rq,
unsigned int hctx_idx, unsigned int request_idx,
unsigned int numa_node)
.map_queue = blk_mq_map_queue,
.queue_rq = scsi_queue_rq,
.complete = scsi_softirq_done,
- .timeout = scsi_times_out,
+ .timeout = scsi_timeout,
.init_request = scsi_init_request,
.exit_request = scsi_exit_request,
};
mutex_unlock(&sd_ref_mutex);
}
-static void sd_prot_op(struct scsi_cmnd *scmd, unsigned int dif)
-{
- unsigned int prot_op = SCSI_PROT_NORMAL;
- unsigned int dix = scsi_prot_sg_count(scmd);
-
- if (scmd->sc_data_direction == DMA_FROM_DEVICE) {
- if (dif && dix)
- prot_op = SCSI_PROT_READ_PASS;
- else if (dif && !dix)
- prot_op = SCSI_PROT_READ_STRIP;
- else if (!dif && dix)
- prot_op = SCSI_PROT_READ_INSERT;
- } else {
- if (dif && dix)
- prot_op = SCSI_PROT_WRITE_PASS;
- else if (dif && !dix)
- prot_op = SCSI_PROT_WRITE_INSERT;
- else if (!dif && dix)
- prot_op = SCSI_PROT_WRITE_STRIP;
+
+
+static unsigned char sd_setup_protect_cmnd(struct scsi_cmnd *scmd,
+ unsigned int dix, unsigned int dif)
+{
+ struct bio *bio = scmd->request->bio;
+ unsigned int prot_op = sd_prot_op(rq_data_dir(scmd->request), dix, dif);
+ unsigned int protect = 0;
+
+ if (dix) { /* DIX Type 0, 1, 2, 3 */
+ if (bio_integrity_flagged(bio, BIP_IP_CHECKSUM))
+ scmd->prot_flags |= SCSI_PROT_IP_CHECKSUM;
+
+ if (bio_integrity_flagged(bio, BIP_CTRL_NOCHECK) == false)
+ scmd->prot_flags |= SCSI_PROT_GUARD_CHECK;
+ }
+
+ if (dif != SD_DIF_TYPE3_PROTECTION) { /* DIX/DIF Type 0, 1, 2 */
+ scmd->prot_flags |= SCSI_PROT_REF_INCREMENT;
+
+ if (bio_integrity_flagged(bio, BIP_CTRL_NOCHECK) == false)
+ scmd->prot_flags |= SCSI_PROT_REF_CHECK;
+ }
+
+ if (dif) { /* DIX/DIF Type 1, 2, 3 */
+ scmd->prot_flags |= SCSI_PROT_TRANSFER_PI;
+
+ if (bio_integrity_flagged(bio, BIP_DISK_NOCHECK))
+ protect = 3 << 5; /* Disable target PI checking */
+ else
+ protect = 1 << 5; /* Enable target PI checking */
}
scsi_set_prot_op(scmd, prot_op);
scsi_set_prot_type(scmd, dif);
+ scmd->prot_flags &= sd_prot_flag_mask(prot_op);
+
+ return protect;
}
static void sd_config_discard(struct scsi_disk *sdkp, unsigned int mode)
sector_t block = blk_rq_pos(rq);
sector_t threshold;
unsigned int this_count = blk_rq_sectors(rq);
- int ret, host_dif;
+ unsigned int dif, dix;
+ int ret;
unsigned char protect;
ret = scsi_init_io(SCpnt, GFP_ATOMIC);
SCpnt->cmnd[0] = WRITE_6;
if (blk_integrity_rq(rq))
- sd_dif_prepare(rq, block, sdp->sector_size);
+ sd_dif_prepare(SCpnt);
} else if (rq_data_dir(rq) == READ) {
SCpnt->cmnd[0] = READ_6;
"writing" : "reading", this_count,
blk_rq_sectors(rq)));
- /* Set RDPROTECT/WRPROTECT if disk is formatted with DIF */
- host_dif = scsi_host_dif_capable(sdp->host, sdkp->protection_type);
- if (host_dif)
- protect = 1 << 5;
+ dix = scsi_prot_sg_count(SCpnt);
+ dif = scsi_host_dif_capable(SCpnt->device->host, sdkp->protection_type);
+
+ if (dif || dix)
+ protect = sd_setup_protect_cmnd(SCpnt, dix, dif);
else
protect = 0;
- if (host_dif == SD_DIF_TYPE2_PROTECTION) {
+ if (protect && sdkp->protection_type == SD_DIF_TYPE2_PROTECTION) {
SCpnt->cmnd = mempool_alloc(sd_cdb_pool, GFP_ATOMIC);
if (unlikely(SCpnt->cmnd == NULL)) {
}
SCpnt->sdb.length = this_count * sdp->sector_size;
- /* If DIF or DIX is enabled, tell HBA how to handle request */
- if (host_dif || scsi_prot_sg_count(SCpnt))
- sd_prot_op(SCpnt, host_dif);
-
/*
* We shouldn't disconnect in the middle of a sector, so with a dumb
* host adapter, it's safe to assume that we can at least transfer
rot = get_unaligned_be16(&buffer[4]);
- if (rot == 1)
+ if (rot == 1) {
queue_flag_set_unlocked(QUEUE_FLAG_NONROT, sdkp->disk->queue);
+ queue_flag_clear_unlocked(QUEUE_FLAG_ADD_RANDOM, sdkp->disk->queue);
+ }
out:
kfree(buffer);
SD_DIF_TYPE3_PROTECTION = 0x3,
};
+/*
+ * Look up the DIX operation based on whether the command is read or
+ * write and whether dix and dif are enabled.
+ */
+static inline unsigned int sd_prot_op(bool write, bool dix, bool dif)
+{
+ /* Lookup table: bit 2 (write), bit 1 (dix), bit 0 (dif) */
+ const unsigned int ops[] = { /* wrt dix dif */
+ SCSI_PROT_NORMAL, /* 0 0 0 */
+ SCSI_PROT_READ_STRIP, /* 0 0 1 */
+ SCSI_PROT_READ_INSERT, /* 0 1 0 */
+ SCSI_PROT_READ_PASS, /* 0 1 1 */
+ SCSI_PROT_NORMAL, /* 1 0 0 */
+ SCSI_PROT_WRITE_INSERT, /* 1 0 1 */
+ SCSI_PROT_WRITE_STRIP, /* 1 1 0 */
+ SCSI_PROT_WRITE_PASS, /* 1 1 1 */
+ };
+
+ return ops[write << 2 | dix << 1 | dif];
+}
+
+/*
+ * Returns a mask of the protection flags that are valid for a given DIX
+ * operation.
+ */
+static inline unsigned int sd_prot_flag_mask(unsigned int prot_op)
+{
+ const unsigned int flag_mask[] = {
+ [SCSI_PROT_NORMAL] = 0,
+
+ [SCSI_PROT_READ_STRIP] = SCSI_PROT_TRANSFER_PI |
+ SCSI_PROT_GUARD_CHECK |
+ SCSI_PROT_REF_CHECK |
+ SCSI_PROT_REF_INCREMENT,
+
+ [SCSI_PROT_READ_INSERT] = SCSI_PROT_REF_INCREMENT |
+ SCSI_PROT_IP_CHECKSUM,
+
+ [SCSI_PROT_READ_PASS] = SCSI_PROT_TRANSFER_PI |
+ SCSI_PROT_GUARD_CHECK |
+ SCSI_PROT_REF_CHECK |
+ SCSI_PROT_REF_INCREMENT |
+ SCSI_PROT_IP_CHECKSUM,
+
+ [SCSI_PROT_WRITE_INSERT] = SCSI_PROT_TRANSFER_PI |
+ SCSI_PROT_REF_INCREMENT,
+
+ [SCSI_PROT_WRITE_STRIP] = SCSI_PROT_GUARD_CHECK |
+ SCSI_PROT_REF_CHECK |
+ SCSI_PROT_REF_INCREMENT |
+ SCSI_PROT_IP_CHECKSUM,
+
+ [SCSI_PROT_WRITE_PASS] = SCSI_PROT_TRANSFER_PI |
+ SCSI_PROT_GUARD_CHECK |
+ SCSI_PROT_REF_CHECK |
+ SCSI_PROT_REF_INCREMENT |
+ SCSI_PROT_IP_CHECKSUM,
+ };
+
+ return flag_mask[prot_op];
+}
+
/*
* Data Integrity Field tuple.
*/
#ifdef CONFIG_BLK_DEV_INTEGRITY
extern void sd_dif_config_host(struct scsi_disk *);
-extern void sd_dif_prepare(struct request *rq, sector_t, unsigned int);
+extern void sd_dif_prepare(struct scsi_cmnd *scmd);
extern void sd_dif_complete(struct scsi_cmnd *, unsigned int);
#else /* CONFIG_BLK_DEV_INTEGRITY */
static inline void sd_dif_config_host(struct scsi_disk *disk)
{
}
-static inline int sd_dif_prepare(struct request *rq, sector_t s, unsigned int a)
+static inline int sd_dif_prepare(struct scsi_cmnd *scmd)
{
return 0;
}
*/
#include <linux/blkdev.h>
-#include <linux/crc-t10dif.h>
+#include <linux/t10-pi.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
#include <scsi/scsi_ioctl.h>
#include <scsi/scsicam.h>
-#include <net/checksum.h>
-
#include "sd.h"
-typedef __u16 (csum_fn) (void *, unsigned int);
-
-static __u16 sd_dif_crc_fn(void *data, unsigned int len)
-{
- return cpu_to_be16(crc_t10dif(data, len));
-}
-
-static __u16 sd_dif_ip_fn(void *data, unsigned int len)
-{
- return ip_compute_csum(data, len);
-}
-
-/*
- * Type 1 and Type 2 protection use the same format: 16 bit guard tag,
- * 16 bit app tag, 32 bit reference tag.
- */
-static void sd_dif_type1_generate(struct blk_integrity_exchg *bix, csum_fn *fn)
-{
- void *buf = bix->data_buf;
- struct sd_dif_tuple *sdt = bix->prot_buf;
- sector_t sector = bix->sector;
- unsigned int i;
-
- for (i = 0 ; i < bix->data_size ; i += bix->sector_size, sdt++) {
- sdt->guard_tag = fn(buf, bix->sector_size);
- sdt->ref_tag = cpu_to_be32(sector & 0xffffffff);
- sdt->app_tag = 0;
-
- buf += bix->sector_size;
- sector++;
- }
-}
-
-static void sd_dif_type1_generate_crc(struct blk_integrity_exchg *bix)
-{
- sd_dif_type1_generate(bix, sd_dif_crc_fn);
-}
-
-static void sd_dif_type1_generate_ip(struct blk_integrity_exchg *bix)
-{
- sd_dif_type1_generate(bix, sd_dif_ip_fn);
-}
-
-static int sd_dif_type1_verify(struct blk_integrity_exchg *bix, csum_fn *fn)
-{
- void *buf = bix->data_buf;
- struct sd_dif_tuple *sdt = bix->prot_buf;
- sector_t sector = bix->sector;
- unsigned int i;
- __u16 csum;
-
- for (i = 0 ; i < bix->data_size ; i += bix->sector_size, sdt++) {
- /* Unwritten sectors */
- if (sdt->app_tag == 0xffff)
- return 0;
-
- if (be32_to_cpu(sdt->ref_tag) != (sector & 0xffffffff)) {
- printk(KERN_ERR
- "%s: ref tag error on sector %lu (rcvd %u)\n",
- bix->disk_name, (unsigned long)sector,
- be32_to_cpu(sdt->ref_tag));
- return -EIO;
- }
-
- csum = fn(buf, bix->sector_size);
-
- if (sdt->guard_tag != csum) {
- printk(KERN_ERR "%s: guard tag error on sector %lu " \
- "(rcvd %04x, data %04x)\n", bix->disk_name,
- (unsigned long)sector,
- be16_to_cpu(sdt->guard_tag), be16_to_cpu(csum));
- return -EIO;
- }
-
- buf += bix->sector_size;
- sector++;
- }
-
- return 0;
-}
-
-static int sd_dif_type1_verify_crc(struct blk_integrity_exchg *bix)
-{
- return sd_dif_type1_verify(bix, sd_dif_crc_fn);
-}
-
-static int sd_dif_type1_verify_ip(struct blk_integrity_exchg *bix)
-{
- return sd_dif_type1_verify(bix, sd_dif_ip_fn);
-}
-
-/*
- * Functions for interleaving and deinterleaving application tags
- */
-static void sd_dif_type1_set_tag(void *prot, void *tag_buf, unsigned int sectors)
-{
- struct sd_dif_tuple *sdt = prot;
- u8 *tag = tag_buf;
- unsigned int i, j;
-
- for (i = 0, j = 0 ; i < sectors ; i++, j += 2, sdt++) {
- sdt->app_tag = tag[j] << 8 | tag[j+1];
- BUG_ON(sdt->app_tag == 0xffff);
- }
-}
-
-static void sd_dif_type1_get_tag(void *prot, void *tag_buf, unsigned int sectors)
-{
- struct sd_dif_tuple *sdt = prot;
- u8 *tag = tag_buf;
- unsigned int i, j;
-
- for (i = 0, j = 0 ; i < sectors ; i++, j += 2, sdt++) {
- tag[j] = (sdt->app_tag & 0xff00) >> 8;
- tag[j+1] = sdt->app_tag & 0xff;
- }
-}
-
-static struct blk_integrity dif_type1_integrity_crc = {
- .name = "T10-DIF-TYPE1-CRC",
- .generate_fn = sd_dif_type1_generate_crc,
- .verify_fn = sd_dif_type1_verify_crc,
- .get_tag_fn = sd_dif_type1_get_tag,
- .set_tag_fn = sd_dif_type1_set_tag,
- .tuple_size = sizeof(struct sd_dif_tuple),
- .tag_size = 0,
-};
-
-static struct blk_integrity dif_type1_integrity_ip = {
- .name = "T10-DIF-TYPE1-IP",
- .generate_fn = sd_dif_type1_generate_ip,
- .verify_fn = sd_dif_type1_verify_ip,
- .get_tag_fn = sd_dif_type1_get_tag,
- .set_tag_fn = sd_dif_type1_set_tag,
- .tuple_size = sizeof(struct sd_dif_tuple),
- .tag_size = 0,
-};
-
-
-/*
- * Type 3 protection has a 16-bit guard tag and 16 + 32 bits of opaque
- * tag space.
- */
-static void sd_dif_type3_generate(struct blk_integrity_exchg *bix, csum_fn *fn)
-{
- void *buf = bix->data_buf;
- struct sd_dif_tuple *sdt = bix->prot_buf;
- unsigned int i;
-
- for (i = 0 ; i < bix->data_size ; i += bix->sector_size, sdt++) {
- sdt->guard_tag = fn(buf, bix->sector_size);
- sdt->ref_tag = 0;
- sdt->app_tag = 0;
-
- buf += bix->sector_size;
- }
-}
-
-static void sd_dif_type3_generate_crc(struct blk_integrity_exchg *bix)
-{
- sd_dif_type3_generate(bix, sd_dif_crc_fn);
-}
-
-static void sd_dif_type3_generate_ip(struct blk_integrity_exchg *bix)
-{
- sd_dif_type3_generate(bix, sd_dif_ip_fn);
-}
-
-static int sd_dif_type3_verify(struct blk_integrity_exchg *bix, csum_fn *fn)
-{
- void *buf = bix->data_buf;
- struct sd_dif_tuple *sdt = bix->prot_buf;
- sector_t sector = bix->sector;
- unsigned int i;
- __u16 csum;
-
- for (i = 0 ; i < bix->data_size ; i += bix->sector_size, sdt++) {
- /* Unwritten sectors */
- if (sdt->app_tag == 0xffff && sdt->ref_tag == 0xffffffff)
- return 0;
-
- csum = fn(buf, bix->sector_size);
-
- if (sdt->guard_tag != csum) {
- printk(KERN_ERR "%s: guard tag error on sector %lu " \
- "(rcvd %04x, data %04x)\n", bix->disk_name,
- (unsigned long)sector,
- be16_to_cpu(sdt->guard_tag), be16_to_cpu(csum));
- return -EIO;
- }
-
- buf += bix->sector_size;
- sector++;
- }
-
- return 0;
-}
-
-static int sd_dif_type3_verify_crc(struct blk_integrity_exchg *bix)
-{
- return sd_dif_type3_verify(bix, sd_dif_crc_fn);
-}
-
-static int sd_dif_type3_verify_ip(struct blk_integrity_exchg *bix)
-{
- return sd_dif_type3_verify(bix, sd_dif_ip_fn);
-}
-
-static void sd_dif_type3_set_tag(void *prot, void *tag_buf, unsigned int sectors)
-{
- struct sd_dif_tuple *sdt = prot;
- u8 *tag = tag_buf;
- unsigned int i, j;
-
- for (i = 0, j = 0 ; i < sectors ; i++, j += 6, sdt++) {
- sdt->app_tag = tag[j] << 8 | tag[j+1];
- sdt->ref_tag = tag[j+2] << 24 | tag[j+3] << 16 |
- tag[j+4] << 8 | tag[j+5];
- }
-}
-
-static void sd_dif_type3_get_tag(void *prot, void *tag_buf, unsigned int sectors)
-{
- struct sd_dif_tuple *sdt = prot;
- u8 *tag = tag_buf;
- unsigned int i, j;
-
- for (i = 0, j = 0 ; i < sectors ; i++, j += 2, sdt++) {
- tag[j] = (sdt->app_tag & 0xff00) >> 8;
- tag[j+1] = sdt->app_tag & 0xff;
- tag[j+2] = (sdt->ref_tag & 0xff000000) >> 24;
- tag[j+3] = (sdt->ref_tag & 0xff0000) >> 16;
- tag[j+4] = (sdt->ref_tag & 0xff00) >> 8;
- tag[j+5] = sdt->ref_tag & 0xff;
- BUG_ON(sdt->app_tag == 0xffff || sdt->ref_tag == 0xffffffff);
- }
-}
-
-static struct blk_integrity dif_type3_integrity_crc = {
- .name = "T10-DIF-TYPE3-CRC",
- .generate_fn = sd_dif_type3_generate_crc,
- .verify_fn = sd_dif_type3_verify_crc,
- .get_tag_fn = sd_dif_type3_get_tag,
- .set_tag_fn = sd_dif_type3_set_tag,
- .tuple_size = sizeof(struct sd_dif_tuple),
- .tag_size = 0,
-};
-
-static struct blk_integrity dif_type3_integrity_ip = {
- .name = "T10-DIF-TYPE3-IP",
- .generate_fn = sd_dif_type3_generate_ip,
- .verify_fn = sd_dif_type3_verify_ip,
- .get_tag_fn = sd_dif_type3_get_tag,
- .set_tag_fn = sd_dif_type3_set_tag,
- .tuple_size = sizeof(struct sd_dif_tuple),
- .tag_size = 0,
-};
-
/*
* Configure exchange of protection information between OS and HBA.
*/
return;
/* Enable DMA of protection information */
- if (scsi_host_get_guard(sdkp->device->host) & SHOST_DIX_GUARD_IP)
+ if (scsi_host_get_guard(sdkp->device->host) & SHOST_DIX_GUARD_IP) {
if (type == SD_DIF_TYPE3_PROTECTION)
- blk_integrity_register(disk, &dif_type3_integrity_ip);
+ blk_integrity_register(disk, &t10_pi_type3_ip);
else
- blk_integrity_register(disk, &dif_type1_integrity_ip);
- else
+ blk_integrity_register(disk, &t10_pi_type1_ip);
+
+ disk->integrity->flags |= BLK_INTEGRITY_IP_CHECKSUM;
+ } else
if (type == SD_DIF_TYPE3_PROTECTION)
- blk_integrity_register(disk, &dif_type3_integrity_crc);
+ blk_integrity_register(disk, &t10_pi_type3_crc);
else
- blk_integrity_register(disk, &dif_type1_integrity_crc);
+ blk_integrity_register(disk, &t10_pi_type1_crc);
sd_printk(KERN_NOTICE, sdkp,
"Enabling DIX %s protection\n", disk->integrity->name);
/* Signal to block layer that we support sector tagging */
- if (dif && type && sdkp->ATO) {
+ if (dif && type) {
+
+ disk->integrity->flags |= BLK_INTEGRITY_DEVICE_CAPABLE;
+
+ if (!sdkp)
+ return;
+
if (type == SD_DIF_TYPE3_PROTECTION)
disk->integrity->tag_size = sizeof(u16) + sizeof(u32);
else
*
* Type 3 does not have a reference tag so no remapping is required.
*/
-void sd_dif_prepare(struct request *rq, sector_t hw_sector,
- unsigned int sector_sz)
+void sd_dif_prepare(struct scsi_cmnd *scmd)
{
- const int tuple_sz = sizeof(struct sd_dif_tuple);
+ const int tuple_sz = sizeof(struct t10_pi_tuple);
struct bio *bio;
struct scsi_disk *sdkp;
- struct sd_dif_tuple *sdt;
+ struct t10_pi_tuple *pi;
u32 phys, virt;
- sdkp = rq->bio->bi_bdev->bd_disk->private_data;
+ sdkp = scsi_disk(scmd->request->rq_disk);
if (sdkp->protection_type == SD_DIF_TYPE3_PROTECTION)
return;
- phys = hw_sector & 0xffffffff;
+ phys = scsi_prot_ref_tag(scmd);
- __rq_for_each_bio(bio, rq) {
+ __rq_for_each_bio(bio, scmd->request) {
+ struct bio_integrity_payload *bip = bio_integrity(bio);
struct bio_vec iv;
struct bvec_iter iter;
unsigned int j;
/* Already remapped? */
- if (bio_flagged(bio, BIO_MAPPED_INTEGRITY))
+ if (bip->bip_flags & BIP_MAPPED_INTEGRITY)
break;
- virt = bio->bi_integrity->bip_iter.bi_sector & 0xffffffff;
+ virt = bip_get_seed(bip) & 0xffffffff;
- bip_for_each_vec(iv, bio->bi_integrity, iter) {
- sdt = kmap_atomic(iv.bv_page)
- + iv.bv_offset;
+ bip_for_each_vec(iv, bip, iter) {
+ pi = kmap_atomic(iv.bv_page) + iv.bv_offset;
- for (j = 0; j < iv.bv_len; j += tuple_sz, sdt++) {
+ for (j = 0; j < iv.bv_len; j += tuple_sz, pi++) {
- if (be32_to_cpu(sdt->ref_tag) == virt)
- sdt->ref_tag = cpu_to_be32(phys);
+ if (be32_to_cpu(pi->ref_tag) == virt)
+ pi->ref_tag = cpu_to_be32(phys);
virt++;
phys++;
}
- kunmap_atomic(sdt);
+ kunmap_atomic(pi);
}
- bio->bi_flags |= (1 << BIO_MAPPED_INTEGRITY);
+ bip->bip_flags |= BIP_MAPPED_INTEGRITY;
}
}
*/
void sd_dif_complete(struct scsi_cmnd *scmd, unsigned int good_bytes)
{
- const int tuple_sz = sizeof(struct sd_dif_tuple);
+ const int tuple_sz = sizeof(struct t10_pi_tuple);
struct scsi_disk *sdkp;
struct bio *bio;
- struct sd_dif_tuple *sdt;
- unsigned int j, sectors, sector_sz;
+ struct t10_pi_tuple *pi;
+ unsigned int j, intervals;
u32 phys, virt;
sdkp = scsi_disk(scmd->request->rq_disk);
if (sdkp->protection_type == SD_DIF_TYPE3_PROTECTION || good_bytes == 0)
return;
- sector_sz = scmd->device->sector_size;
- sectors = good_bytes / sector_sz;
-
- phys = blk_rq_pos(scmd->request) & 0xffffffff;
- if (sector_sz == 4096)
- phys >>= 3;
+ intervals = good_bytes / scsi_prot_interval(scmd);
+ phys = scsi_prot_ref_tag(scmd);
__rq_for_each_bio(bio, scmd->request) {
+ struct bio_integrity_payload *bip = bio_integrity(bio);
struct bio_vec iv;
struct bvec_iter iter;
- virt = bio->bi_integrity->bip_iter.bi_sector & 0xffffffff;
+ virt = bip_get_seed(bip) & 0xffffffff;
- bip_for_each_vec(iv, bio->bi_integrity, iter) {
- sdt = kmap_atomic(iv.bv_page)
- + iv.bv_offset;
+ bip_for_each_vec(iv, bip, iter) {
+ pi = kmap_atomic(iv.bv_page) + iv.bv_offset;
- for (j = 0; j < iv.bv_len; j += tuple_sz, sdt++) {
+ for (j = 0; j < iv.bv_len; j += tuple_sz, pi++) {
- if (sectors == 0) {
- kunmap_atomic(sdt);
+ if (intervals == 0) {
+ kunmap_atomic(pi);
return;
}
- if (be32_to_cpu(sdt->ref_tag) == phys)
- sdt->ref_tag = cpu_to_be32(virt);
+ if (be32_to_cpu(pi->ref_tag) == phys)
+ pi->ref_tag = cpu_to_be32(virt);
virt++;
phys++;
- sectors--;
+ intervals--;
}
- kunmap_atomic(sdt);
+ kunmap_atomic(pi);
}
}
}
}
rq = blk_get_request(q, rw, GFP_ATOMIC);
- if (!rq) {
+ if (IS_ERR(rq)) {
kfree(long_cmdp);
- return -ENOMEM;
+ return PTR_ERR(rq);
}
blk_rq_set_block_pc(rq);
req = blk_get_request(SRpnt->stp->device->request_queue, write,
GFP_KERNEL);
- if (!req)
+ if (IS_ERR(req))
return DRIVER_ERROR << 24;
blk_rq_set_block_pc(req);
/* CPU hotplug notifier */
struct notifier_block nb;
+ /* Protected by event_vq lock */
+ bool stop_events;
+
struct virtio_scsi_vq ctrl_vq;
struct virtio_scsi_vq event_vq;
struct virtio_scsi_vq req_vqs[];
{
int i;
+ /* Stop scheduling work before calling cancel_work_sync. */
+ spin_lock_irq(&vscsi->event_vq.vq_lock);
+ vscsi->stop_events = true;
+ spin_unlock_irq(&vscsi->event_vq.vq_lock);
+
for (i = 0; i < VIRTIO_SCSI_EVENT_LEN; i++)
cancel_work_sync(&vscsi->event_list[i].work);
}
{
struct virtio_scsi_event_node *event_node = buf;
- schedule_work(&event_node->work);
+ if (!vscsi->stop_events)
+ queue_work(system_freezable_wq, &event_node->work);
}
static void virtscsi_event_done(struct virtqueue *vq)
virtscsi_vq->vq = vq;
}
-static void virtscsi_scan(struct virtio_device *vdev)
-{
- struct Scsi_Host *shost = (struct Scsi_Host *)vdev->priv;
-
- scsi_scan_host(shost);
-}
-
static void virtscsi_remove_vqs(struct virtio_device *vdev)
{
struct Scsi_Host *sh = virtio_scsi_host(vdev);
virtscsi_config_set(vdev, cdb_size, VIRTIO_SCSI_CDB_SIZE);
virtscsi_config_set(vdev, sense_size, VIRTIO_SCSI_SENSE_SIZE);
- if (virtio_has_feature(vdev, VIRTIO_SCSI_F_HOTPLUG))
- virtscsi_kick_event_all(vscsi);
-
err = 0;
out:
err = scsi_add_host(shost, &vdev->dev);
if (err)
goto scsi_add_host_failed;
- /*
- * scsi_scan_host() happens in virtscsi_scan() via virtio_driver->scan()
- * after VIRTIO_CONFIG_S_DRIVER_OK has been set..
- */
+
+ virtio_device_ready(vdev);
+
+ if (virtio_has_feature(vdev, VIRTIO_SCSI_F_HOTPLUG))
+ virtscsi_kick_event_all(vscsi);
+
+ scsi_scan_host(shost);
return 0;
scsi_add_host_failed:
return err;
err = register_hotcpu_notifier(&vscsi->nb);
- if (err)
+ if (err) {
vdev->config->del_vqs(vdev);
+ return err;
+ }
+
+ virtio_device_ready(vdev);
+
+ if (virtio_has_feature(vdev, VIRTIO_SCSI_F_HOTPLUG))
+ virtscsi_kick_event_all(vscsi);
return err;
}
.driver.owner = THIS_MODULE,
.id_table = id_table,
.probe = virtscsi_probe,
- .scan = virtscsi_scan,
#ifdef CONFIG_PM_SLEEP
.freeze = virtscsi_freeze,
.restore = virtscsi_restore,
req = blk_get_request(pdv->pdv_sd->request_queue,
(data_direction == DMA_TO_DEVICE),
GFP_KERNEL);
- if (!req) {
+ if (IS_ERR(req)) {
pr_err("PSCSI: blk_get_request() failed\n");
ret = TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE;
goto fail;
}
EXPORT_SYMBOL_GPL(virtio_check_driver_offered_feature);
+static void __virtio_config_changed(struct virtio_device *dev)
+{
+ struct virtio_driver *drv = drv_to_virtio(dev->dev.driver);
+
+ if (!dev->config_enabled)
+ dev->config_change_pending = true;
+ else if (drv && drv->config_changed)
+ drv->config_changed(dev);
+}
+
+void virtio_config_changed(struct virtio_device *dev)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->config_lock, flags);
+ __virtio_config_changed(dev);
+ spin_unlock_irqrestore(&dev->config_lock, flags);
+}
+EXPORT_SYMBOL_GPL(virtio_config_changed);
+
+static void virtio_config_disable(struct virtio_device *dev)
+{
+ spin_lock_irq(&dev->config_lock);
+ dev->config_enabled = false;
+ spin_unlock_irq(&dev->config_lock);
+}
+
+static void virtio_config_enable(struct virtio_device *dev)
+{
+ spin_lock_irq(&dev->config_lock);
+ dev->config_enabled = true;
+ if (dev->config_change_pending)
+ __virtio_config_changed(dev);
+ dev->config_change_pending = false;
+ spin_unlock_irq(&dev->config_lock);
+}
+
static int virtio_dev_probe(struct device *_d)
{
int err, i;
add_status(dev, VIRTIO_CONFIG_S_DRIVER_OK);
if (drv->scan)
drv->scan(dev);
+
+ virtio_config_enable(dev);
}
return err;
struct virtio_device *dev = dev_to_virtio(_d);
struct virtio_driver *drv = drv_to_virtio(dev->dev.driver);
+ virtio_config_disable(dev);
+
drv->remove(dev);
/* Driver should have reset device. */
dev->index = err;
dev_set_name(&dev->dev, "virtio%u", dev->index);
+ spin_lock_init(&dev->config_lock);
+ dev->config_enabled = false;
+ dev->config_change_pending = false;
+
/* We always start by resetting the device, in case a previous
* driver messed it up. This also tests that code path a little. */
dev->config->reset(dev);
}
EXPORT_SYMBOL_GPL(unregister_virtio_device);
+#ifdef CONFIG_PM_SLEEP
+int virtio_device_freeze(struct virtio_device *dev)
+{
+ struct virtio_driver *drv = drv_to_virtio(dev->dev.driver);
+
+ virtio_config_disable(dev);
+
+ dev->failed = dev->config->get_status(dev) & VIRTIO_CONFIG_S_FAILED;
+
+ if (drv && drv->freeze)
+ return drv->freeze(dev);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(virtio_device_freeze);
+
+int virtio_device_restore(struct virtio_device *dev)
+{
+ struct virtio_driver *drv = drv_to_virtio(dev->dev.driver);
+
+ /* We always start by resetting the device, in case a previous
+ * driver messed it up. */
+ dev->config->reset(dev);
+
+ /* Acknowledge that we've seen the device. */
+ add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE);
+
+ /* Maybe driver failed before freeze.
+ * Restore the failed status, for debugging. */
+ if (dev->failed)
+ add_status(dev, VIRTIO_CONFIG_S_FAILED);
+
+ if (!drv)
+ return 0;
+
+ /* We have a driver! */
+ add_status(dev, VIRTIO_CONFIG_S_DRIVER);
+
+ dev->config->finalize_features(dev);
+
+ if (drv->restore) {
+ int ret = drv->restore(dev);
+ if (ret) {
+ add_status(dev, VIRTIO_CONFIG_S_FAILED);
+ return ret;
+ }
+ }
+
+ /* Finally, tell the device we're all set */
+ add_status(dev, VIRTIO_CONFIG_S_DRIVER_OK);
+
+ virtio_config_enable(dev);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(virtio_device_restore);
+#endif
+
static int virtio_init(void)
{
if (bus_register(&virtio_bus) != 0)
if (ret)
return ret;
+ virtio_device_ready(vdev);
+
fill_balloon(vb, towards_target(vb));
update_balloon_size(vb);
return 0;
{
struct virtio_mmio_device *vm_dev = opaque;
struct virtio_mmio_vq_info *info;
- struct virtio_driver *vdrv = container_of(vm_dev->vdev.dev.driver,
- struct virtio_driver, driver);
unsigned long status;
unsigned long flags;
irqreturn_t ret = IRQ_NONE;
status = readl(vm_dev->base + VIRTIO_MMIO_INTERRUPT_STATUS);
writel(status, vm_dev->base + VIRTIO_MMIO_INTERRUPT_ACK);
- if (unlikely(status & VIRTIO_MMIO_INT_CONFIG)
- && vdrv && vdrv->config_changed) {
- vdrv->config_changed(&vm_dev->vdev);
+ if (unlikely(status & VIRTIO_MMIO_INT_CONFIG)) {
+ virtio_config_changed(&vm_dev->vdev);
ret = IRQ_HANDLED;
}
/* Vectors allocated, excluding per-vq vectors if any */
unsigned msix_used_vectors;
- /* Status saved during hibernate/restore */
- u8 saved_status;
-
/* Whether we have vector per vq */
bool per_vq_vectors;
};
static irqreturn_t vp_config_changed(int irq, void *opaque)
{
struct virtio_pci_device *vp_dev = opaque;
- struct virtio_driver *drv;
- drv = container_of(vp_dev->vdev.dev.driver,
- struct virtio_driver, driver);
- if (drv && drv->config_changed)
- drv->config_changed(&vp_dev->vdev);
+ virtio_config_changed(&vp_dev->vdev);
return IRQ_HANDLED;
}
{
struct pci_dev *pci_dev = to_pci_dev(dev);
struct virtio_pci_device *vp_dev = pci_get_drvdata(pci_dev);
- struct virtio_driver *drv;
int ret;
- drv = container_of(vp_dev->vdev.dev.driver,
- struct virtio_driver, driver);
-
- ret = 0;
- vp_dev->saved_status = vp_get_status(&vp_dev->vdev);
- if (drv && drv->freeze)
- ret = drv->freeze(&vp_dev->vdev);
+ ret = virtio_device_freeze(&vp_dev->vdev);
if (!ret)
pci_disable_device(pci_dev);
{
struct pci_dev *pci_dev = to_pci_dev(dev);
struct virtio_pci_device *vp_dev = pci_get_drvdata(pci_dev);
- struct virtio_driver *drv;
int ret;
- drv = container_of(vp_dev->vdev.dev.driver,
- struct virtio_driver, driver);
-
ret = pci_enable_device(pci_dev);
if (ret)
return ret;
pci_set_master(pci_dev);
- vp_finalize_features(&vp_dev->vdev);
-
- if (drv && drv->restore)
- ret = drv->restore(&vp_dev->vdev);
-
- /* Finally, tell the device we're all set */
- if (!ret)
- vp_set_status(&vp_dev->vdev, vp_dev->saved_status);
-
- return ret;
+ return virtio_device_restore(&vp_dev->vdev);
}
static const struct dev_pm_ops virtio_pci_pm_ops = {
EXPORT_SYMBOL(I_BDEV);
/*
- * Move the inode from its current bdi to a new bdi. If the inode is dirty we
- * need to move it onto the dirty list of @dst so that the inode is always on
- * the right list.
+ * Move the inode from its current bdi to a new bdi. Make sure the inode
+ * is clean before moving so that it doesn't linger on the old bdi.
*/
static void bdev_inode_switch_bdi(struct inode *inode,
struct backing_dev_info *dst)
{
- struct backing_dev_info *old = inode->i_data.backing_dev_info;
- bool wakeup_bdi = false;
-
- if (unlikely(dst == old)) /* deadlock avoidance */
- return;
- bdi_lock_two(&old->wb, &dst->wb);
- spin_lock(&inode->i_lock);
- inode->i_data.backing_dev_info = dst;
- if (inode->i_state & I_DIRTY) {
- if (bdi_cap_writeback_dirty(dst) && !wb_has_dirty_io(&dst->wb))
- wakeup_bdi = true;
- list_move(&inode->i_wb_list, &dst->wb.b_dirty);
+ while (true) {
+ spin_lock(&inode->i_lock);
+ if (!(inode->i_state & I_DIRTY)) {
+ inode->i_data.backing_dev_info = dst;
+ spin_unlock(&inode->i_lock);
+ return;
+ }
+ spin_unlock(&inode->i_lock);
+ WARN_ON_ONCE(write_inode_now(inode, true));
}
- spin_unlock(&inode->i_lock);
- spin_unlock(&old->wb.list_lock);
- spin_unlock(&dst->wb.list_lock);
-
- if (wakeup_bdi)
- bdi_wakeup_thread_delayed(dst);
}
/* Kill _all_ buffers and pagecache , dirty or not.. */
if (!ret) {
bd_set_size(bdev,(loff_t)get_capacity(disk)<<9);
bdi = blk_get_backing_dev_info(bdev);
- if (bdi == NULL)
- bdi = &default_backing_dev_info;
bdev_inode_switch_bdi(bdev->bd_inode, bdi);
}
if (!device->bdev)
continue;
bdi = blk_get_backing_dev_info(device->bdev);
- if (bdi && bdi_congested(bdi, bdi_bits)) {
+ if (bdi_congested(bdi, bdi_bits)) {
ret = 1;
break;
}
nfsv4-$(CONFIG_NFS_USE_LEGACY_DNS) += cache_lib.o
nfsv4-$(CONFIG_SYSCTL) += nfs4sysctl.o
nfsv4-$(CONFIG_NFS_V4_1) += pnfs.o pnfs_dev.o
+nfsv4-$(CONFIG_NFS_V4_2) += nfs42proc.o
obj-$(CONFIG_PNFS_FILE_LAYOUT) += filelayout/
obj-$(CONFIG_PNFS_OBJLAYOUT) += objlayout/
#else
VM_BUG_ON(iocb->ki_nbytes != PAGE_SIZE);
- if (rw == READ || rw == KERNEL_READ)
- return nfs_file_direct_read(iocb, iter, pos,
- rw == READ ? true : false);
- return nfs_file_direct_write(iocb, iter, pos,
- rw == WRITE ? true : false);
+ if (rw == READ)
+ return nfs_file_direct_read(iocb, iter, pos);
+ return nfs_file_direct_write(iocb, iter, pos);
#endif /* CONFIG_NFS_SWAP */
}
* cache.
*/
ssize_t nfs_file_direct_read(struct kiocb *iocb, struct iov_iter *iter,
- loff_t pos, bool uio)
+ loff_t pos)
{
struct file *file = iocb->ki_filp;
struct address_space *mapping = file->f_mapping;
* is no atomic O_APPEND write facility in the NFS protocol.
*/
ssize_t nfs_file_direct_write(struct kiocb *iocb, struct iov_iter *iter,
- loff_t pos, bool uio)
+ loff_t pos)
{
ssize_t result = -EINVAL;
struct file *file = iocb->ki_filp;
ssize_t result;
if (iocb->ki_filp->f_flags & O_DIRECT)
- return nfs_file_direct_read(iocb, to, iocb->ki_pos, true);
+ return nfs_file_direct_read(iocb, to, iocb->ki_pos);
dprintk("NFS: read(%pD2, %zu@%lu)\n",
iocb->ki_filp,
return result;
if (file->f_flags & O_DIRECT)
- return nfs_file_direct_write(iocb, from, pos, true);
+ return nfs_file_direct_write(iocb, from, pos);
dprintk("NFS: write(%pD2, %zu@%Ld)\n",
file, count, (long long) pos);
}
out:
nfs_request_remove_commit_list(req, cinfo);
- pnfs_put_lseg_async(freeme);
+ pnfs_put_lseg_locked(freeme);
}
static void
kfree(new);
return res;
}
+EXPORT_SYMBOL_GPL(nfs_get_lock_context);
void nfs_put_lock_context(struct nfs_lock_context *l_ctx)
{
spin_unlock(&inode->i_lock);
kfree(l_ctx);
}
+EXPORT_SYMBOL_GPL(nfs_put_lock_context);
/**
* nfs_close_context - Common close_context() routine NFSv2/v3
--- /dev/null
+/*
+ * Copyright (c) 2014 Anna Schumaker <Anna.Schumaker@Netapp.com>
+ */
+
+#ifndef __LINUX_FS_NFS_NFS4_2_H
+#define __LINUX_FS_NFS_NFS4_2_H
+
+/* nfs4.2proc.c */
+loff_t nfs42_proc_llseek(struct file *, loff_t, int);
+
+/* nfs4.2xdr.h */
+extern struct rpc_procinfo nfs4_2_procedures[];
+
+#endif /* __LINUX_FS_NFS_NFS4_2_H */
--- /dev/null
+/*
+ * Copyright (c) 2014 Anna Schumaker <Anna.Schumaker@Netapp.com>
+ */
+#include <linux/fs.h>
+#include <linux/sunrpc/sched.h>
+#include <linux/nfs.h>
+#include <linux/nfs3.h>
+#include <linux/nfs4.h>
+#include <linux/nfs_xdr.h>
+#include <linux/nfs_fs.h>
+#include "nfs4_fs.h"
+#include "nfs42.h"
+
+static int nfs42_set_rw_stateid(nfs4_stateid *dst, struct file *file,
+ fmode_t fmode)
+{
+ struct nfs_open_context *open;
+ struct nfs_lock_context *lock;
+ int ret;
+
+ open = get_nfs_open_context(nfs_file_open_context(file));
+ lock = nfs_get_lock_context(open);
+ if (IS_ERR(lock)) {
+ put_nfs_open_context(open);
+ return PTR_ERR(lock);
+ }
+
+ ret = nfs4_set_rw_stateid(dst, open, lock, fmode);
+
+ nfs_put_lock_context(lock);
+ put_nfs_open_context(open);
+ return ret;
+}
+
+loff_t nfs42_proc_llseek(struct file *filep, loff_t offset, int whence)
+{
+ struct inode *inode = file_inode(filep);
+ struct nfs42_seek_args args = {
+ .sa_fh = NFS_FH(inode),
+ .sa_offset = offset,
+ .sa_what = (whence == SEEK_HOLE) ?
+ NFS4_CONTENT_HOLE : NFS4_CONTENT_DATA,
+ };
+ struct nfs42_seek_res res;
+ struct rpc_message msg = {
+ .rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_SEEK],
+ .rpc_argp = &args,
+ .rpc_resp = &res,
+ };
+ struct nfs_server *server = NFS_SERVER(inode);
+ int status;
+
+ if (!(server->caps & NFS_CAP_SEEK))
+ return -ENOTSUPP;
+
+ status = nfs42_set_rw_stateid(&args.sa_stateid, filep, FMODE_READ);
+ if (status)
+ return status;
+
+ nfs_wb_all(inode);
+ status = nfs4_call_sync(server->client, server, &msg,
+ &args.seq_args, &res.seq_res, 0);
+ if (status == -ENOTSUPP)
+ server->caps &= ~NFS_CAP_SEEK;
+ if (status)
+ return status;
+
+ return vfs_setpos(filep, res.sr_offset, inode->i_sb->s_maxbytes);
+}
--- /dev/null
+/*
+ * Copyright (c) 2014 Anna Schumaker <Anna.Schumaker@Netapp.com>
+ */
+#ifndef __LINUX_FS_NFS_NFS4_2XDR_H
+#define __LINUX_FS_NFS_NFS4_2XDR_H
+
+#define encode_seek_maxsz (op_encode_hdr_maxsz + \
+ encode_stateid_maxsz + \
+ 2 /* offset */ + \
+ 1 /* whence */)
+#define decode_seek_maxsz (op_decode_hdr_maxsz + \
+ 1 /* eof */ + \
+ 1 /* whence */ + \
+ 2 /* offset */ + \
+ 2 /* length */)
+
+#define NFS4_enc_seek_sz (compound_encode_hdr_maxsz + \
+ encode_putfh_maxsz + \
+ encode_seek_maxsz)
+#define NFS4_dec_seek_sz (compound_decode_hdr_maxsz + \
+ decode_putfh_maxsz + \
+ decode_seek_maxsz)
+
+
+static void encode_seek(struct xdr_stream *xdr,
+ struct nfs42_seek_args *args,
+ struct compound_hdr *hdr)
+{
+ encode_op_hdr(xdr, OP_SEEK, decode_seek_maxsz, hdr);
+ encode_nfs4_stateid(xdr, &args->sa_stateid);
+ encode_uint64(xdr, args->sa_offset);
+ encode_uint32(xdr, args->sa_what);
+}
+
+/*
+ * Encode SEEK request
+ */
+static void nfs4_xdr_enc_seek(struct rpc_rqst *req,
+ struct xdr_stream *xdr,
+ struct nfs42_seek_args *args)
+{
+ struct compound_hdr hdr = {
+ .minorversion = nfs4_xdr_minorversion(&args->seq_args),
+ };
+
+ encode_compound_hdr(xdr, req, &hdr);
+ encode_sequence(xdr, &args->seq_args, &hdr);
+ encode_putfh(xdr, args->sa_fh, &hdr);
+ encode_seek(xdr, args, &hdr);
+ encode_nops(&hdr);
+}
+
+static int decode_seek(struct xdr_stream *xdr, struct nfs42_seek_res *res)
+{
+ int status;
+ __be32 *p;
+
+ status = decode_op_hdr(xdr, OP_SEEK);
+ if (status)
+ return status;
+
+ p = xdr_inline_decode(xdr, 4 + 8);
+ if (unlikely(!p))
+ goto out_overflow;
+
+ res->sr_eof = be32_to_cpup(p++);
+ p = xdr_decode_hyper(p, &res->sr_offset);
+ return 0;
+
+out_overflow:
+ print_overflow_msg(__func__, xdr);
+ return -EIO;
+}
+
+/*
+ * Decode SEEK request
+ */
+static int nfs4_xdr_dec_seek(struct rpc_rqst *rqstp,
+ struct xdr_stream *xdr,
+ struct nfs42_seek_res *res)
+{
+ struct compound_hdr hdr;
+ int status;
+
+ status = decode_compound_hdr(xdr, &hdr);
+ if (status)
+ goto out;
+ status = decode_sequence(xdr, &res->seq_res, rqstp);
+ if (status)
+ goto out;
+ status = decode_putfh(xdr);
+ if (status)
+ goto out;
+ status = decode_seek(xdr, res);
+out:
+ return status;
+}
+#endif /* __LINUX_FS_NFS_NFS4_2XDR_H */
const struct nfs4_fs_locations *locations);
/* nfs4proc.c */
+extern int nfs4_call_sync(struct rpc_clnt *, struct nfs_server *,
+ struct rpc_message *, struct nfs4_sequence_args *,
+ struct nfs4_sequence_res *, int);
extern int nfs4_proc_setclientid(struct nfs_client *, u32, unsigned short, struct rpc_cred *, struct nfs4_setclientid_res *);
extern int nfs4_proc_setclientid_confirm(struct nfs_client *, struct nfs4_setclientid_res *arg, struct rpc_cred *);
extern int nfs4_proc_get_rootfh(struct nfs_server *, struct nfs_fh *, struct nfs_fsinfo *, bool);
#include "fscache.h"
#include "pnfs.h"
+#ifdef CONFIG_NFS_V4_2
+#include "nfs42.h"
+#endif
+
#define NFSDBG_FACILITY NFSDBG_FILE
static int
return ret;
}
+#ifdef CONFIG_NFS_V4_2
+static loff_t nfs4_file_llseek(struct file *filep, loff_t offset, int whence)
+{
+ loff_t ret;
+
+ switch (whence) {
+ case SEEK_HOLE:
+ case SEEK_DATA:
+ ret = nfs42_proc_llseek(filep, offset, whence);
+ if (ret != -ENOTSUPP)
+ return ret;
+ default:
+ return nfs_file_llseek(filep, offset, whence);
+ }
+}
+#endif /* CONFIG_NFS_V4_2 */
+
const struct file_operations nfs4_file_operations = {
+#ifdef CONFIG_NFS_V4_2
+ .llseek = nfs4_file_llseek,
+#else
.llseek = nfs_file_llseek,
+#endif
.read = new_sync_read,
.write = new_sync_write,
.read_iter = nfs_file_read,
return ret;
}
-static
int nfs4_call_sync(struct rpc_clnt *clnt,
struct nfs_server *server,
struct rpc_message *msg,
| NFS_CAP_CHANGE_ATTR
| NFS_CAP_POSIX_LOCK
| NFS_CAP_STATEID_NFSV41
- | NFS_CAP_ATOMIC_OPEN_V1,
+ | NFS_CAP_ATOMIC_OPEN_V1
+ | NFS_CAP_SEEK,
.init_client = nfs41_init_client,
.shutdown_client = nfs41_shutdown_client,
.match_stateid = nfs41_match_stateid,
return -stat;
}
+#ifdef CONFIG_NFS_V4_2
+#include "nfs42xdr.c"
+#endif /* CONFIG_NFS_V4_2 */
+
#define PROC(proc, argtype, restype) \
[NFSPROC4_CLNT_##proc] = { \
.p_proc = NFSPROC4_COMPOUND, \
enc_bind_conn_to_session, dec_bind_conn_to_session),
PROC(DESTROY_CLIENTID, enc_destroy_clientid, dec_destroy_clientid),
#endif /* CONFIG_NFS_V4_1 */
+#ifdef CONFIG_NFS_V4_2
+ PROC(SEEK, enc_seek, dec_seek),
+#endif /* CONFIG_NFS_V4_2 */
};
const struct rpc_version nfs_version4 = {
*/
void nfs_pgio_data_destroy(struct nfs_pgio_header *hdr)
{
- put_nfs_open_context(hdr->args.context);
+ if (hdr->args.context)
+ put_nfs_open_context(hdr->args.context);
if (hdr->page_array.pagevec != hdr->page_array.page_array)
kfree(hdr->page_array.pagevec);
}
nfs_list_remove_request(req);
nfs_list_add_request(req, &hdr->pages);
- if (WARN_ON_ONCE(pageused >= pagecount))
- return nfs_pgio_error(desc, hdr);
-
if (!last_page || last_page != req->wb_page) {
- *pages++ = last_page = req->wb_page;
pageused++;
+ if (pageused > pagecount)
+ break;
+ *pages++ = last_page = req->wb_page;
}
}
if (WARN_ON_ONCE(pageused != pagecount))
}
EXPORT_SYMBOL_GPL(pnfs_put_lseg);
-static void pnfs_put_lseg_async_work(struct work_struct *work)
+static void pnfs_free_lseg_async_work(struct work_struct *work)
{
struct pnfs_layout_segment *lseg;
+ struct pnfs_layout_hdr *lo;
lseg = container_of(work, struct pnfs_layout_segment, pls_work);
+ lo = lseg->pls_layout;
- pnfs_put_lseg(lseg);
+ pnfs_free_lseg(lseg);
+ pnfs_put_layout_hdr(lo);
}
-void
-pnfs_put_lseg_async(struct pnfs_layout_segment *lseg)
+static void pnfs_free_lseg_async(struct pnfs_layout_segment *lseg)
{
- INIT_WORK(&lseg->pls_work, pnfs_put_lseg_async_work);
+ INIT_WORK(&lseg->pls_work, pnfs_free_lseg_async_work);
schedule_work(&lseg->pls_work);
}
-EXPORT_SYMBOL_GPL(pnfs_put_lseg_async);
+
+void
+pnfs_put_lseg_locked(struct pnfs_layout_segment *lseg)
+{
+ if (!lseg)
+ return;
+
+ assert_spin_locked(&lseg->pls_layout->plh_inode->i_lock);
+
+ dprintk("%s: lseg %p ref %d valid %d\n", __func__, lseg,
+ atomic_read(&lseg->pls_refcount),
+ test_bit(NFS_LSEG_VALID, &lseg->pls_flags));
+ if (atomic_dec_and_test(&lseg->pls_refcount)) {
+ struct pnfs_layout_hdr *lo = lseg->pls_layout;
+ pnfs_get_layout_hdr(lo);
+ pnfs_layout_remove_lseg(lo, lseg);
+ pnfs_free_lseg_async(lseg);
+ }
+}
+EXPORT_SYMBOL_GPL(pnfs_put_lseg_locked);
static u64
end_offset(u64 start, u64 len)
/* pnfs.c */
void pnfs_get_layout_hdr(struct pnfs_layout_hdr *lo);
void pnfs_put_lseg(struct pnfs_layout_segment *lseg);
-void pnfs_put_lseg_async(struct pnfs_layout_segment *lseg);
+void pnfs_put_lseg_locked(struct pnfs_layout_segment *lseg);
void set_pnfs_layoutdriver(struct nfs_server *, const struct nfs_fh *, u32);
void unset_pnfs_layoutdriver(struct nfs_server *);
void nfs4_delete_deviceid(const struct pnfs_layoutdriver_type *, const struct nfs_client *, const struct nfs4_deviceid *);
void nfs4_init_deviceid_node(struct nfs4_deviceid_node *, struct nfs_server *,
const struct nfs4_deviceid *);
-struct nfs4_deviceid_node *nfs4_insert_deviceid_node(struct nfs4_deviceid_node *);
bool nfs4_put_deviceid_node(struct nfs4_deviceid_node *);
void nfs4_mark_deviceid_unavailable(struct nfs4_deviceid_node *node);
bool nfs4_test_deviceid_unavailable(struct nfs4_deviceid_node *node);
{
}
-static inline void pnfs_put_lseg_async(struct pnfs_layout_segment *lseg)
-{
-}
-
static inline int pnfs_return_layout(struct inode *ino)
{
return 0;
btp->bt_dev = bdev->bd_dev;
btp->bt_bdev = bdev;
btp->bt_bdi = blk_get_backing_dev_info(bdev);
- if (!btp->bt_bdi)
- goto error;
if (xfs_setsize_buftarg_early(btp, bdev))
goto error;
* Bits in backing_dev_info.state
*/
enum bdi_state {
- BDI_wb_alloc, /* Default embedded wb allocated */
BDI_async_congested, /* The async (write) queue is getting full */
BDI_sync_congested, /* The sync queue is getting full */
BDI_registered, /* bdi_register() was done */
BDI_writeback_running, /* Writeback is in progress */
- BDI_unused, /* Available bits start here */
};
typedef int (congested_fn)(void *, int);
struct bdi_writeback {
struct backing_dev_info *bdi; /* our parent bdi */
- unsigned int nr;
unsigned long last_old_flush; /* last old data flush */
void bdi_writeback_workfn(struct work_struct *work);
int bdi_has_dirty_io(struct backing_dev_info *bdi);
void bdi_wakeup_thread_delayed(struct backing_dev_info *bdi);
-void bdi_lock_two(struct bdi_writeback *wb1, struct bdi_writeback *wb2);
extern spinlock_t bdi_lock;
extern struct list_head bdi_list;
*/
#define bio_get(bio) atomic_inc(&(bio)->bi_cnt)
+enum bip_flags {
+ BIP_BLOCK_INTEGRITY = 1 << 0, /* block layer owns integrity data */
+ BIP_MAPPED_INTEGRITY = 1 << 1, /* ref tag has been remapped */
+ BIP_CTRL_NOCHECK = 1 << 2, /* disable HBA integrity checking */
+ BIP_DISK_NOCHECK = 1 << 3, /* disable disk integrity checking */
+ BIP_IP_CHECKSUM = 1 << 4, /* IP checksum */
+};
+
#if defined(CONFIG_BLK_DEV_INTEGRITY)
+
+static inline struct bio_integrity_payload *bio_integrity(struct bio *bio)
+{
+ if (bio->bi_rw & REQ_INTEGRITY)
+ return bio->bi_integrity;
+
+ return NULL;
+}
+
/*
* bio integrity payload
*/
struct bvec_iter bip_iter;
- /* kill - should just use bip_vec */
- void *bip_buf; /* generated integrity data */
-
bio_end_io_t *bip_end_io; /* saved I/O completion fn */
unsigned short bip_slab; /* slab the bip came from */
unsigned short bip_vcnt; /* # of integrity bio_vecs */
unsigned short bip_max_vcnt; /* integrity bio_vec slots */
- unsigned bip_owns_buf:1; /* should free bip_buf */
+ unsigned short bip_flags; /* control flags */
struct work_struct bip_work; /* I/O completion */
struct bio_vec *bip_vec;
struct bio_vec bip_inline_vecs[0];/* embedded bvec array */
};
+
+static inline bool bio_integrity_flagged(struct bio *bio, enum bip_flags flag)
+{
+ struct bio_integrity_payload *bip = bio_integrity(bio);
+
+ if (bip)
+ return bip->bip_flags & flag;
+
+ return false;
+}
+
+static inline sector_t bip_get_seed(struct bio_integrity_payload *bip)
+{
+ return bip->bip_iter.bi_sector;
+}
+
+static inline void bip_set_seed(struct bio_integrity_payload *bip,
+ sector_t seed)
+{
+ bip->bip_iter.bi_sector = seed;
+}
+
#endif /* CONFIG_BLK_DEV_INTEGRITY */
extern void bio_trim(struct bio *bio, int offset, int size);
}
extern struct bio_set *bioset_create(unsigned int, unsigned int);
+extern struct bio_set *bioset_create_nobvec(unsigned int, unsigned int);
extern void bioset_free(struct bio_set *);
extern mempool_t *biovec_create_pool(int pool_entries);
extern struct bio *bio_clone_bioset(struct bio *, gfp_t, struct bio_set *bs);
extern struct bio_set *fs_bio_set;
-unsigned int bio_integrity_tag_size(struct bio *bio);
static inline struct bio *bio_alloc(gfp_t gfp_mask, unsigned int nr_iovecs)
{
for_each_bio(_bio) \
bip_for_each_vec(_bvl, _bio->bi_integrity, _iter)
-#define bio_integrity(bio) (bio->bi_integrity != NULL)
-
extern struct bio_integrity_payload *bio_integrity_alloc(struct bio *, gfp_t, unsigned int);
extern void bio_integrity_free(struct bio *);
extern int bio_integrity_add_page(struct bio *, struct page *, unsigned int, unsigned int);
-extern int bio_integrity_enabled(struct bio *bio);
-extern int bio_integrity_set_tag(struct bio *, void *, unsigned int);
-extern int bio_integrity_get_tag(struct bio *, void *, unsigned int);
+extern bool bio_integrity_enabled(struct bio *bio);
extern int bio_integrity_prep(struct bio *);
extern void bio_integrity_endio(struct bio *, int);
extern void bio_integrity_advance(struct bio *, unsigned int);
#else /* CONFIG_BLK_DEV_INTEGRITY */
-static inline int bio_integrity(struct bio *bio)
+static inline void *bio_integrity(struct bio *bio)
{
- return 0;
+ return NULL;
}
-static inline int bio_integrity_enabled(struct bio *bio)
+static inline bool bio_integrity_enabled(struct bio *bio)
{
- return 0;
+ return false;
}
static inline int bioset_integrity_create(struct bio_set *bs, int pool_size)
return;
}
+static inline bool bio_integrity_flagged(struct bio *bio, enum bip_flags flag)
+{
+ return false;
+}
+
#endif /* CONFIG_BLK_DEV_INTEGRITY */
#endif /* CONFIG_BLOCK */
#include <linux/blkdev.h>
struct blk_mq_tags;
+struct blk_flush_queue;
struct blk_mq_cpu_notifier {
struct list_head list;
struct request_queue *queue;
unsigned int queue_num;
+ struct blk_flush_queue *fq;
void *driver_data;
struct list_head tag_list;
};
-typedef int (queue_rq_fn)(struct blk_mq_hw_ctx *, struct request *);
+typedef int (queue_rq_fn)(struct blk_mq_hw_ctx *, struct request *, bool);
typedef struct blk_mq_hw_ctx *(map_queue_fn)(struct request_queue *, const int);
+typedef enum blk_eh_timer_return (timeout_fn)(struct request *, bool);
typedef int (init_hctx_fn)(struct blk_mq_hw_ctx *, void *, unsigned int);
typedef void (exit_hctx_fn)(struct blk_mq_hw_ctx *, unsigned int);
typedef int (init_request_fn)(void *, struct request *, unsigned int,
typedef void (exit_request_fn)(void *, struct request *, unsigned int,
unsigned int);
+typedef void (busy_iter_fn)(struct blk_mq_hw_ctx *, struct request *, void *,
+ bool);
+
struct blk_mq_ops {
/*
* Queue request
/*
* Called on request timeout
*/
- rq_timed_out_fn *timeout;
+ timeout_fn *timeout;
softirq_done_fn *complete;
/*
* Called for every command allocated by the block layer to allow
* the driver to set up driver specific data.
+ *
+ * Tag greater than or equal to queue_depth is for setting up
+ * flush request.
+ *
* Ditto for exit/teardown.
*/
init_request_fn *init_request;
struct blk_mq_hw_ctx *blk_mq_map_queue(struct request_queue *, const int ctx_index);
struct blk_mq_hw_ctx *blk_mq_alloc_single_hw_queue(struct blk_mq_tag_set *, unsigned int, int);
-void blk_mq_end_io(struct request *rq, int error);
-void __blk_mq_end_io(struct request *rq, int error);
+void blk_mq_start_request(struct request *rq);
+void blk_mq_end_request(struct request *rq, int error);
+void __blk_mq_end_request(struct request *rq, int error);
void blk_mq_requeue_request(struct request *rq);
void blk_mq_add_to_requeue_list(struct request *rq, bool at_head);
void blk_mq_start_hw_queues(struct request_queue *q);
void blk_mq_start_stopped_hw_queues(struct request_queue *q, bool async);
void blk_mq_delay_queue(struct blk_mq_hw_ctx *hctx, unsigned long msecs);
-void blk_mq_tag_busy_iter(struct blk_mq_tags *tags, void (*fn)(void *data, unsigned long *), void *data);
+void blk_mq_tag_busy_iter(struct blk_mq_hw_ctx *hctx, busy_iter_fn *fn,
+ void *priv);
/*
* Driver command data is immediately after the request. So subtract request
struct io_context *bi_ioc;
struct cgroup_subsys_state *bi_css;
#endif
+ union {
#if defined(CONFIG_BLK_DEV_INTEGRITY)
- struct bio_integrity_payload *bi_integrity; /* data integrity */
+ struct bio_integrity_payload *bi_integrity; /* data integrity */
#endif
+ };
unsigned short bi_vcnt; /* how many bio_vec's */
#define BIO_USER_MAPPED 6 /* contains user pages */
#define BIO_EOPNOTSUPP 7 /* not supported */
#define BIO_NULL_MAPPED 8 /* contains invalid user pages */
-#define BIO_FS_INTEGRITY 9 /* fs owns integrity data, not block layer */
-#define BIO_QUIET 10 /* Make BIO Quiet */
-#define BIO_MAPPED_INTEGRITY 11/* integrity metadata has been remapped */
-#define BIO_SNAP_STABLE 12 /* bio data must be snapshotted during write */
+#define BIO_QUIET 9 /* Make BIO Quiet */
+#define BIO_SNAP_STABLE 10 /* bio data must be snapshotted during write */
/*
* Flags starting here get preserved by bio_reset() - this includes
__REQ_WRITE_SAME, /* write same block many times */
__REQ_NOIDLE, /* don't anticipate more IO after this one */
+ __REQ_INTEGRITY, /* I/O includes block integrity payload */
__REQ_FUA, /* forced unit access */
__REQ_FLUSH, /* request for cache flush */
__REQ_FLUSH_SEQ, /* request for flush sequence */
__REQ_IO_STAT, /* account I/O stat */
__REQ_MIXED_MERGE, /* merge of different types, fail separately */
- __REQ_KERNEL, /* direct IO to kernel pages */
__REQ_PM, /* runtime pm request */
- __REQ_END, /* last of chain of requests */
__REQ_HASHED, /* on IO scheduler merge hash */
__REQ_MQ_INFLIGHT, /* track inflight for MQ */
__REQ_NR_BITS, /* stops here */
#define REQ_DISCARD (1ULL << __REQ_DISCARD)
#define REQ_WRITE_SAME (1ULL << __REQ_WRITE_SAME)
#define REQ_NOIDLE (1ULL << __REQ_NOIDLE)
+#define REQ_INTEGRITY (1ULL << __REQ_INTEGRITY)
#define REQ_FAILFAST_MASK \
(REQ_FAILFAST_DEV | REQ_FAILFAST_TRANSPORT | REQ_FAILFAST_DRIVER)
#define REQ_COMMON_MASK \
(REQ_WRITE | REQ_FAILFAST_MASK | REQ_SYNC | REQ_META | REQ_PRIO | \
REQ_DISCARD | REQ_WRITE_SAME | REQ_NOIDLE | REQ_FLUSH | REQ_FUA | \
- REQ_SECURE)
+ REQ_SECURE | REQ_INTEGRITY)
#define REQ_CLONE_MASK REQ_COMMON_MASK
#define BIO_NO_ADVANCE_ITER_MASK (REQ_DISCARD|REQ_WRITE_SAME)
#define REQ_IO_STAT (1ULL << __REQ_IO_STAT)
#define REQ_MIXED_MERGE (1ULL << __REQ_MIXED_MERGE)
#define REQ_SECURE (1ULL << __REQ_SECURE)
-#define REQ_KERNEL (1ULL << __REQ_KERNEL)
#define REQ_PM (1ULL << __REQ_PM)
-#define REQ_END (1ULL << __REQ_END)
#define REQ_HASHED (1ULL << __REQ_HASHED)
#define REQ_MQ_INFLIGHT (1ULL << __REQ_MQ_INFLIGHT)
struct sg_io_hdr;
struct bsg_job;
struct blkcg_gq;
+struct blk_flush_queue;
#define BLKDEV_MIN_RQ 4
#define BLKDEV_MAX_RQ 128 /* Default maximum */
*/
unsigned int flush_flags;
unsigned int flush_not_queueable:1;
- unsigned int flush_queue_delayed:1;
- unsigned int flush_pending_idx:1;
- unsigned int flush_running_idx:1;
- unsigned long flush_pending_since;
- struct list_head flush_queue[2];
- struct list_head flush_data_in_flight;
- struct request *flush_rq;
- spinlock_t mq_flush_lock;
+ struct blk_flush_queue *fq;
struct list_head requeue_list;
spinlock_t requeue_lock;
static inline struct request_queue *bdev_get_queue(struct block_device *bdev)
{
- return bdev->bd_disk->queue;
+ return bdev->bd_disk->queue; /* this is never NULL */
}
/*
static inline int queue_limit_alignment_offset(struct queue_limits *lim, sector_t sector)
{
unsigned int granularity = max(lim->physical_block_size, lim->io_min);
- unsigned int alignment = (sector << 9) & (granularity - 1);
+ unsigned int alignment = sector_div(sector, granularity >> 9) << 9;
- return (granularity + lim->alignment_offset - alignment)
- & (granularity - 1);
+ return (granularity + lim->alignment_offset - alignment) % granularity;
}
static inline int bdev_alignment_offset(struct block_device *bdev)
#if defined(CONFIG_BLK_DEV_INTEGRITY)
-#define INTEGRITY_FLAG_READ 2 /* verify data integrity on read */
-#define INTEGRITY_FLAG_WRITE 4 /* generate data integrity on write */
+enum blk_integrity_flags {
+ BLK_INTEGRITY_VERIFY = 1 << 0,
+ BLK_INTEGRITY_GENERATE = 1 << 1,
+ BLK_INTEGRITY_DEVICE_CAPABLE = 1 << 2,
+ BLK_INTEGRITY_IP_CHECKSUM = 1 << 3,
+};
-struct blk_integrity_exchg {
+struct blk_integrity_iter {
void *prot_buf;
void *data_buf;
- sector_t sector;
+ sector_t seed;
unsigned int data_size;
- unsigned short sector_size;
+ unsigned short interval;
const char *disk_name;
};
-typedef void (integrity_gen_fn) (struct blk_integrity_exchg *);
-typedef int (integrity_vrfy_fn) (struct blk_integrity_exchg *);
-typedef void (integrity_set_tag_fn) (void *, void *, unsigned int);
-typedef void (integrity_get_tag_fn) (void *, void *, unsigned int);
+typedef int (integrity_processing_fn) (struct blk_integrity_iter *);
struct blk_integrity {
- integrity_gen_fn *generate_fn;
- integrity_vrfy_fn *verify_fn;
- integrity_set_tag_fn *set_tag_fn;
- integrity_get_tag_fn *get_tag_fn;
+ integrity_processing_fn *generate_fn;
+ integrity_processing_fn *verify_fn;
unsigned short flags;
unsigned short tuple_size;
- unsigned short sector_size;
+ unsigned short interval;
unsigned short tag_size;
const char *name;
extern int blk_rq_map_integrity_sg(struct request_queue *, struct bio *,
struct scatterlist *);
extern int blk_rq_count_integrity_sg(struct request_queue *, struct bio *);
-extern int blk_integrity_merge_rq(struct request_queue *, struct request *,
- struct request *);
-extern int blk_integrity_merge_bio(struct request_queue *, struct request *,
- struct bio *);
+extern bool blk_integrity_merge_rq(struct request_queue *, struct request *,
+ struct request *);
+extern bool blk_integrity_merge_bio(struct request_queue *, struct request *,
+ struct bio *);
static inline
struct blk_integrity *bdev_get_integrity(struct block_device *bdev)
return disk->integrity;
}
-static inline int blk_integrity_rq(struct request *rq)
+static inline bool blk_integrity_rq(struct request *rq)
{
- if (rq->bio == NULL)
- return 0;
-
- return bio_integrity(rq->bio);
+ return rq->cmd_flags & REQ_INTEGRITY;
}
static inline void blk_queue_max_integrity_segments(struct request_queue *q,
{
return 0;
}
-static inline int blk_integrity_merge_rq(struct request_queue *rq,
- struct request *r1,
- struct request *r2)
+static inline bool blk_integrity_merge_rq(struct request_queue *rq,
+ struct request *r1,
+ struct request *r2)
{
return 0;
}
-static inline int blk_integrity_merge_bio(struct request_queue *rq,
- struct request *r,
- struct bio *b)
+static inline bool blk_integrity_merge_bio(struct request_queue *rq,
+ struct request *r,
+ struct bio *b)
{
return 0;
}
#define CRC_T10DIF_DIGEST_SIZE 2
#define CRC_T10DIF_BLOCK_SIZE 1
-__u16 crc_t10dif_generic(__u16 crc, const unsigned char *buffer, size_t len);
-__u16 crc_t10dif(unsigned char const *, size_t);
+extern __u16 crc_t10dif_generic(__u16 crc, const unsigned char *buffer,
+ size_t len);
+extern __u16 crc_t10dif(unsigned char const *, size_t);
#endif
#define READ 0
#define WRITE RW_MASK
#define READA RWA_MASK
-#define KERNEL_READ (READ|REQ_KERNEL)
-#define KERNEL_WRITE (WRITE|REQ_KERNEL)
#define READ_SYNC (READ | REQ_SYNC)
#define WRITE_SYNC (WRITE | REQ_SYNC | REQ_NOIDLE)
#define __assign(attr_nr, attr_flag, name, nla_type, type, assignment...) \
nla = ntb[attr_nr]; \
if (nla) { \
- if (exclude_invariants && ((attr_flag) & DRBD_F_INVARIANT)) { \
+ if (exclude_invariants && !!((attr_flag) & DRBD_F_INVARIANT)) { \
pr_info("<< must not change invariant attr: %s\n", #name); \
return -EEXIST; \
} \
assignment; \
- } else if (exclude_invariants && ((attr_flag) & DRBD_F_INVARIANT)) { \
+ } else if (exclude_invariants && !!((attr_flag) & DRBD_F_INVARIANT)) { \
/* attribute missing from payload, */ \
/* which was expected */ \
} else if ((attr_flag) & DRBD_F_REQUIRED) { \
u16 capi20_get_version(u32 contr, struct capi_version *verp);
u16 capi20_get_serial(u32 contr, u8 serial[CAPI_SERIAL_LEN]);
u16 capi20_get_profile(u32 contr, struct capi_profile *profp);
-int capi20_manufacturer(unsigned int cmd, void __user *data);
+int capi20_manufacturer(unsigned long cmd, void __user *data);
#define CAPICTR_UP 0
#define CAPICTR_DOWN 1
uint16_t MaxBufWriteSize;
uint8_t NumEraseRegions;
uint32_t EraseRegionInfo[0]; /* Not host ordered */
-} __attribute__((packed));
+} __packed;
/* Extended Query Structure for both PRI and ALT */
uint8_t pri[3];
uint8_t MajorVersion;
uint8_t MinorVersion;
-} __attribute__((packed));
+} __packed;
/* Vendor-Specific PRI for Intel/Sharp Extended Command Set (0x0001) */
uint8_t FactProtRegSize;
uint8_t UserProtRegSize;
uint8_t extra[0];
-} __attribute__((packed));
+} __packed;
struct cfi_intelext_otpinfo {
uint32_t ProtRegAddr;
uint8_t FactProtRegSize;
uint16_t UserGroups;
uint8_t UserProtRegSize;
-} __attribute__((packed));
+} __packed;
struct cfi_intelext_blockinfo {
uint16_t NumIdentBlocks;
uint16_t MinBlockEraseCycles;
uint8_t BitsPerCell;
uint8_t BlockCap;
-} __attribute__((packed));
+} __packed;
struct cfi_intelext_regioninfo {
uint16_t NumIdentPartitions;
uint8_t NumOpAllowedSimEraMode;
uint8_t NumBlockTypes;
struct cfi_intelext_blockinfo BlockTypes[1];
-} __attribute__((packed));
+} __packed;
struct cfi_intelext_programming_regioninfo {
uint8_t ProgRegShift;
uint8_t Reserved2;
uint8_t ControlInvalid;
uint8_t Reserved3;
-} __attribute__((packed));
+} __packed;
/* Vendor-Specific PRI for AMD/Fujitsu Extended Command Set (0x0002) */
uint8_t VppMin;
uint8_t VppMax;
uint8_t TopBottom;
-} __attribute__((packed));
+} __packed;
/* Vendor-Specific PRI for Atmel chips (command set 0x0002) */
uint8_t BottomBoot;
uint8_t BurstMode;
uint8_t PageMode;
-} __attribute__((packed));
+} __packed;
struct cfi_pri_query {
uint8_t NumFields;
uint32_t ProtField[1]; /* Not host ordered */
-} __attribute__((packed));
+} __packed;
struct cfi_bri_query {
uint8_t PageModeReadCap;
uint8_t NumFields;
uint32_t ConfField[1]; /* Not host ordered */
-} __attribute__((packed));
+} __packed;
#define P_ID_NONE 0x0000
#define P_ID_INTEL_EXT 0x0001
* @ecc_step_ds: [INTERN] ECC step required by the @ecc_strength_ds,
* also from the datasheet. It is the recommended ECC step
* size, if known; if unknown, set to zero.
+ * @onfi_timing_mode_default: [INTERN] default ONFI timing mode. This field is
+ * either deduced from the datasheet if the NAND
+ * chip is not ONFI compliant or set to 0 if it is
+ * (an ONFI chip is always configured in mode 0
+ * after a NAND reset)
* @numchips: [INTERN] number of physical chips
* @chipsize: [INTERN] the size of one chip for multichip arrays
* @pagemask: [INTERN] page number mask = number of (pages / chip) - 1
uint8_t bits_per_cell;
uint16_t ecc_strength_ds;
uint16_t ecc_step_ds;
+ int onfi_timing_mode_default;
int badblockpos;
int badblockbits;
* @options: stores various chip bit options
* @id_len: The valid length of the @id.
* @oobsize: OOB size
+ * @ecc: ECC correctability and step information from the datasheet.
* @ecc.strength_ds: The ECC correctability from the datasheet, same as the
* @ecc_strength_ds in nand_chip{}.
* @ecc.step_ds: The ECC step required by the @ecc.strength_ds, same as the
* @ecc_step_ds in nand_chip{}, also from the datasheet.
* For example, the "4bit ECC for each 512Byte" can be set with
* NAND_ECC_INFO(4, 512).
+ * @onfi_timing_mode_default: the default ONFI timing mode entered after a NAND
+ * reset. Should be deduced from timings described
+ * in the datasheet.
+ *
*/
struct nand_flash_dev {
char *name;
uint16_t strength_ds;
uint16_t step_ds;
} ecc;
+ int onfi_timing_mode_default;
};
/**
* Callback to use for xmit over the accelerated station. This
* is used in place of ndo_start_xmit on accelerated net
* devices.
+ * bool (*ndo_gso_check) (struct sk_buff *skb,
+ * struct net_device *dev);
+ * Called by core transmit path to determine if device is capable of
+ * performing GSO on a packet. The device returns true if it is
+ * able to GSO the packet, false otherwise. If the return value is
+ * false the stack will do software GSO.
*/
struct net_device_ops {
int (*ndo_init)(struct net_device *dev);
struct net_device *dev,
void *priv);
int (*ndo_get_lock_subclass)(struct net_device *dev);
+ bool (*ndo_gso_check) (struct sk_buff *skb,
+ struct net_device *dev);
};
/**
(!skb_has_frag_list(skb) || (features & NETIF_F_FRAGLIST));
}
-static inline bool netif_needs_gso(struct sk_buff *skb,
+static inline bool netif_needs_gso(struct net_device *dev, struct sk_buff *skb,
netdev_features_t features)
{
return skb_is_gso(skb) && (!skb_gso_ok(skb, features) ||
+ (dev->netdev_ops->ndo_gso_check &&
+ !dev->netdev_ops->ndo_gso_check(skb, dev)) ||
unlikely((skb->ip_summed != CHECKSUM_PARTIAL) &&
(skb->ip_summed != CHECKSUM_UNNECESSARY)));
}
NFSPROC4_CLNT_GETDEVICELIST,
NFSPROC4_CLNT_BIND_CONN_TO_SESSION,
NFSPROC4_CLNT_DESTROY_CLIENTID,
+
+ /* nfs42 */
+ NFSPROC4_CLNT_SEEK,
};
/* nfs41 types */
extern ssize_t nfs_direct_IO(int, struct kiocb *, struct iov_iter *, loff_t);
extern ssize_t nfs_file_direct_read(struct kiocb *iocb,
struct iov_iter *iter,
- loff_t pos, bool uio);
+ loff_t pos);
extern ssize_t nfs_file_direct_write(struct kiocb *iocb,
struct iov_iter *iter,
- loff_t pos, bool uio);
+ loff_t pos);
/*
* linux/fs/nfs/dir.c
#define NFS_CAP_STATEID_NFSV41 (1U << 16)
#define NFS_CAP_ATOMIC_OPEN_V1 (1U << 17)
#define NFS_CAP_SECURITY_LABEL (1U << 18)
+#define NFS_CAP_SEEK (1U << 19)
#endif
#endif /* CONFIG_NFS_V4_1 */
+#ifdef CONFIG_NFS_V4_2
+struct nfs42_seek_args {
+ struct nfs4_sequence_args seq_args;
+
+ struct nfs_fh *sa_fh;
+ nfs4_stateid sa_stateid;
+ u64 sa_offset;
+ u32 sa_what;
+};
+
+struct nfs42_seek_res {
+ struct nfs4_sequence_res seq_res;
+ unsigned int status;
+
+ u32 sr_eof;
+ u64 sr_offset;
+};
+#endif
+
struct nfs_page;
#define NFS_PAGEVEC_SIZE (8U)
int error_loc[16];
};
+#if IS_ENABLED(CONFIG_MTD_NAND_OMAP_BCH)
void elm_decode_bch_error_page(struct device *dev, u8 *ecc_calc,
struct elm_errorvec *err_vec);
int elm_config(struct device *dev, enum bch_ecc bch_type,
int ecc_steps, int ecc_step_size, int ecc_syndrome_size);
+#else
+static inline void
+elm_decode_bch_error_page(struct device *dev, u8 *ecc_calc,
+ struct elm_errorvec *err_vec)
+{
+}
+
+static inline int elm_config(struct device *dev, enum bch_ecc bch_type,
+ int ecc_steps, int ecc_step_size,
+ int ecc_syndrome_size)
+{
+ return -ENOSYS;
+}
+#endif /* CONFIG_MTD_NAND_ECC_BCH */
+
#endif /* __ELM_H */
struct mtd_partition *parts;
int nr_parts;
bool dev_ready;
+ bool flash_bbt;
enum nand_io xfer_type;
int devsize;
enum omap_ecc ecc_opt;
might_sleep_if(pri & __GFP_WAIT);
if (skb_cloned(skb)) {
struct sk_buff *nskb = skb_copy(skb, pri);
- kfree_skb(skb); /* Free our shared copy */
+
+ /* Free our shared copy */
+ if (likely(nskb))
+ consume_skb(skb);
+ else
+ kfree_skb(skb);
skb = nskb;
}
return skb;
--- /dev/null
+#ifndef _LINUX_T10_PI_H
+#define _LINUX_T10_PI_H
+
+#include <linux/types.h>
+#include <linux/blkdev.h>
+
+/*
+ * T10 Protection Information tuple.
+ */
+struct t10_pi_tuple {
+ __be16 guard_tag; /* Checksum */
+ __be16 app_tag; /* Opaque storage */
+ __be32 ref_tag; /* Target LBA or indirect LBA */
+};
+
+
+extern struct blk_integrity t10_pi_type1_crc;
+extern struct blk_integrity t10_pi_type1_ip;
+extern struct blk_integrity t10_pi_type3_crc;
+extern struct blk_integrity t10_pi_type3_ip;
+
+#endif
/**
* virtio_device - representation of a device using virtio
* @index: unique position on the virtio bus
+ * @failed: saved value for CONFIG_S_FAILED bit (for restore)
+ * @config_enabled: configuration change reporting enabled
+ * @config_change_pending: configuration change reported while disabled
+ * @config_lock: protects configuration change reporting
* @dev: underlying device.
* @id: the device type identification (used to match it with a driver).
* @config: the configuration ops for this device.
*/
struct virtio_device {
int index;
+ bool failed;
+ bool config_enabled;
+ bool config_change_pending;
+ spinlock_t config_lock;
struct device dev;
struct virtio_device_id id;
const struct virtio_config_ops *config;
void virtio_break_device(struct virtio_device *dev);
+void virtio_config_changed(struct virtio_device *dev);
+#ifdef CONFIG_PM_SLEEP
+int virtio_device_freeze(struct virtio_device *dev);
+int virtio_device_restore(struct virtio_device *dev);
+#endif
+
/**
* virtio_driver - operations for a virtio I/O driver
* @driver: underlying device driver (populate name and owner).
return vq;
}
+/**
+ * virtio_device_ready - enable vq use in probe function
+ * @vdev: the device
+ *
+ * Driver must call this to use vqs in the probe function.
+ *
+ * Note: vqs are enabled automatically after probe returns.
+ */
+static inline
+void virtio_device_ready(struct virtio_device *dev)
+{
+ unsigned status = dev->config->get_status(dev);
+
+ BUG_ON(status & VIRTIO_CONFIG_S_DRIVER_OK);
+ dev->config->set_status(dev, status | VIRTIO_CONFIG_S_DRIVER_OK);
+}
+
static inline
const char *virtio_bus_name(struct virtio_device *vdev)
{
#include <linux/of.h>
#include <linux/phy.h>
#include <linux/phy_fixed.h>
+#include <linux/ethtool.h>
enum dsa_tag_protocol {
DSA_TAG_PROTO_NONE = 0,
struct ifacaddr6 *aca_next;
int aca_users;
atomic_t aca_refcnt;
- spinlock_t aca_lock;
unsigned long aca_cstamp;
unsigned long aca_tstamp;
};
static inline struct sock *__inet6_lookup_skb(struct inet_hashinfo *hashinfo,
struct sk_buff *skb,
const __be16 sport,
- const __be16 dport)
+ const __be16 dport,
+ int iif)
{
struct sock *sk = skb_steal_sock(skb);
return __inet6_lookup(dev_net(skb_dst(skb)->dev), hashinfo,
&ipv6_hdr(skb)->saddr, sport,
&ipv6_hdr(skb)->daddr, ntohs(dport),
- inet6_iif(skb));
+ iif);
}
struct sock *inet6_lookup(struct net *net, struct inet_hashinfo *hashinfo,
/**
* nlmsg_put - Add a new netlink message to an skb
* @skb: socket buffer to store message in
- * @portid: netlink process id
+ * @portid: netlink PORTID of requesting application
* @seq: sequence number of message
* @type: message type
* @payload: length of message payload
asoc->pmtu_pending = 0;
}
+static inline bool sctp_chunk_pending(const struct sctp_chunk *chunk)
+{
+ return !list_empty(&chunk->list);
+}
+
/* Walk through a list of TLV parameters. Don't trust the
* individual parameter lengths and instead depend on
* the chunk length to indicate when to stop. Make sure
int, __be16);
struct sctp_chunk *sctp_make_asconf_set_prim(struct sctp_association *asoc,
union sctp_addr *addr);
-int sctp_verify_asconf(const struct sctp_association *asoc,
- struct sctp_paramhdr *param_hdr, void *chunk_end,
- struct sctp_paramhdr **errp);
+bool sctp_verify_asconf(const struct sctp_association *asoc,
+ struct sctp_chunk *chunk, bool addr_param_needed,
+ struct sctp_paramhdr **errp);
struct sctp_chunk *sctp_process_asconf(struct sctp_association *asoc,
struct sctp_chunk *asconf);
int sctp_process_asconf_ack(struct sctp_association *asoc,
/* From syncookies.c */
int __cookie_v4_check(const struct iphdr *iph, const struct tcphdr *th,
u32 cookie);
-struct sock *cookie_v4_check(struct sock *sk, struct sk_buff *skb,
- struct ip_options *opt);
+struct sock *cookie_v4_check(struct sock *sk, struct sk_buff *skb);
#ifdef CONFIG_SYN_COOKIES
/* Syncookies use a monotonic timer which increments every 60 seconds.
#define TCP_SKB_CB(__skb) ((struct tcp_skb_cb *)&((__skb)->cb[0]))
+
+/* This is the variant of inet6_iif() that must be used by TCP,
+ * as TCP moves IP6CB into a different location in skb->cb[]
+ */
+static inline int tcp_v6_iif(const struct sk_buff *skb)
+{
+ return TCP_SKB_CB(skb)->header.h6.iif;
+}
+
/* Due to TSO, an SKB can be composed of multiple actual
* packets. To keep these tracked properly, we use this.
*/
void tcp_v4_init(void);
void tcp_init(void);
+/*
+ * Save and compile IPv4 options, return a pointer to it
+ */
+static inline struct ip_options_rcu *tcp_v4_save_options(struct sk_buff *skb)
+{
+ const struct ip_options *opt = &TCP_SKB_CB(skb)->header.h4.opt;
+ struct ip_options_rcu *dopt = NULL;
+
+ if (opt->optlen) {
+ int opt_size = sizeof(*dopt) + opt->optlen;
+
+ dopt = kmalloc(opt_size, GFP_ATOMIC);
+ if (dopt && __ip_options_echo(&dopt->opt, skb, opt)) {
+ kfree(dopt);
+ dopt = NULL;
+ }
+ }
+ return dopt;
+}
+
#endif /* _TCP_H */
#include <scsi/scsi_device.h>
struct Scsi_Host;
-struct scsi_device;
struct scsi_driver;
+#include <scsi/scsi_device.h>
+
/*
* MAX_COMMAND_SIZE is:
* The longest fixed-length SCSI CDB as per the SCSI standard.
unsigned char prot_op;
unsigned char prot_type;
+ unsigned char prot_flags;
unsigned short cmd_len;
enum dma_data_direction sc_data_direction;
return scmd->prot_op;
}
+enum scsi_prot_flags {
+ SCSI_PROT_TRANSFER_PI = 1 << 0,
+ SCSI_PROT_GUARD_CHECK = 1 << 1,
+ SCSI_PROT_REF_CHECK = 1 << 2,
+ SCSI_PROT_REF_INCREMENT = 1 << 3,
+ SCSI_PROT_IP_CHECKSUM = 1 << 4,
+};
+
/*
* The controller usually does not know anything about the target it
* is communicating with. However, when DIX is enabled the controller
return blk_rq_pos(scmd->request);
}
+static inline unsigned int scsi_prot_interval(struct scsi_cmnd *scmd)
+{
+ return scmd->device->sector_size;
+}
+
+static inline u32 scsi_prot_ref_tag(struct scsi_cmnd *scmd)
+{
+ return blk_rq_pos(scmd->request) >>
+ (ilog2(scsi_prot_interval(scmd)) - 9) & 0xffffffff;
+}
+
static inline unsigned scsi_prot_sg_count(struct scsi_cmnd *cmd)
{
return cmd->prot_sdb ? cmd->prot_sdb->table.nents : 0;
static inline unsigned scsi_transfer_length(struct scsi_cmnd *scmd)
{
unsigned int xfer_len = scsi_out(scmd)->length;
- unsigned int prot_op = scsi_get_prot_op(scmd);
- unsigned int sector_size = scmd->device->sector_size;
+ unsigned int prot_interval = scsi_prot_interval(scmd);
- switch (prot_op) {
- case SCSI_PROT_NORMAL:
- case SCSI_PROT_WRITE_STRIP:
- case SCSI_PROT_READ_INSERT:
- return xfer_len;
- }
+ if (scmd->prot_flags & SCSI_PROT_TRANSFER_PI)
+ xfer_len += (xfer_len >> ilog2(prot_interval)) * 8;
- return xfer_len + (xfer_len >> ilog2(sector_size)) * 8;
+ return xfer_len;
}
#endif /* _SCSI_SCSI_CMND_H */
header-y += blkpg.h
header-y += blktrace_api.h
header-y += bpf.h
+header-y += bpf_common.h
header-y += bpqether.h
header-y += bsg.h
header-y += btrfs.h
#define _UAPI__LINUX_BPF_H__
#include <linux/types.h>
+#include <linux/bpf_common.h>
/* Extended instruction set based on top of classic BPF */
--- /dev/null
+#ifndef _UAPI__LINUX_BPF_COMMON_H__
+#define _UAPI__LINUX_BPF_COMMON_H__
+
+/* Instruction classes */
+#define BPF_CLASS(code) ((code) & 0x07)
+#define BPF_LD 0x00
+#define BPF_LDX 0x01
+#define BPF_ST 0x02
+#define BPF_STX 0x03
+#define BPF_ALU 0x04
+#define BPF_JMP 0x05
+#define BPF_RET 0x06
+#define BPF_MISC 0x07
+
+/* ld/ldx fields */
+#define BPF_SIZE(code) ((code) & 0x18)
+#define BPF_W 0x00
+#define BPF_H 0x08
+#define BPF_B 0x10
+#define BPF_MODE(code) ((code) & 0xe0)
+#define BPF_IMM 0x00
+#define BPF_ABS 0x20
+#define BPF_IND 0x40
+#define BPF_MEM 0x60
+#define BPF_LEN 0x80
+#define BPF_MSH 0xa0
+
+/* alu/jmp fields */
+#define BPF_OP(code) ((code) & 0xf0)
+#define BPF_ADD 0x00
+#define BPF_SUB 0x10
+#define BPF_MUL 0x20
+#define BPF_DIV 0x30
+#define BPF_OR 0x40
+#define BPF_AND 0x50
+#define BPF_LSH 0x60
+#define BPF_RSH 0x70
+#define BPF_NEG 0x80
+#define BPF_MOD 0x90
+#define BPF_XOR 0xa0
+
+#define BPF_JA 0x00
+#define BPF_JEQ 0x10
+#define BPF_JGT 0x20
+#define BPF_JGE 0x30
+#define BPF_JSET 0x40
+#define BPF_SRC(code) ((code) & 0x08)
+#define BPF_K 0x00
+#define BPF_X 0x08
+
+#ifndef BPF_MAXINSNS
+#define BPF_MAXINSNS 4096
+#endif
+
+#endif /* _UAPI__LINUX_BPF_COMMON_H__ */
#define DM_DEV_SET_GEOMETRY _IOWR(DM_IOCTL, DM_DEV_SET_GEOMETRY_CMD, struct dm_ioctl)
#define DM_VERSION_MAJOR 4
-#define DM_VERSION_MINOR 27
+#define DM_VERSION_MINOR 28
#define DM_VERSION_PATCHLEVEL 0
-#define DM_VERSION_EXTRA "-ioctl (2013-10-30)"
+#define DM_VERSION_EXTRA "-ioctl (2014-09-17)"
/* Status bits */
#define DM_READONLY_FLAG (1 << 0) /* In/Out */
#include <linux/compiler.h>
#include <linux/types.h>
-
+#include <linux/bpf_common.h>
/*
* Current version of the filter code architecture.
struct sock_filter __user *filter;
};
-/*
- * Instruction classes
- */
-
-#define BPF_CLASS(code) ((code) & 0x07)
-#define BPF_LD 0x00
-#define BPF_LDX 0x01
-#define BPF_ST 0x02
-#define BPF_STX 0x03
-#define BPF_ALU 0x04
-#define BPF_JMP 0x05
-#define BPF_RET 0x06
-#define BPF_MISC 0x07
-
-/* ld/ldx fields */
-#define BPF_SIZE(code) ((code) & 0x18)
-#define BPF_W 0x00
-#define BPF_H 0x08
-#define BPF_B 0x10
-#define BPF_MODE(code) ((code) & 0xe0)
-#define BPF_IMM 0x00
-#define BPF_ABS 0x20
-#define BPF_IND 0x40
-#define BPF_MEM 0x60
-#define BPF_LEN 0x80
-#define BPF_MSH 0xa0
-
-/* alu/jmp fields */
-#define BPF_OP(code) ((code) & 0xf0)
-#define BPF_ADD 0x00
-#define BPF_SUB 0x10
-#define BPF_MUL 0x20
-#define BPF_DIV 0x30
-#define BPF_OR 0x40
-#define BPF_AND 0x50
-#define BPF_LSH 0x60
-#define BPF_RSH 0x70
-#define BPF_NEG 0x80
-#define BPF_MOD 0x90
-#define BPF_XOR 0xa0
-
-#define BPF_JA 0x00
-#define BPF_JEQ 0x10
-#define BPF_JGT 0x20
-#define BPF_JGE 0x30
-#define BPF_JSET 0x40
-#define BPF_SRC(code) ((code) & 0x08)
-#define BPF_K 0x00
-#define BPF_X 0x08
-
/* ret - BPF_K and BPF_X also apply */
#define BPF_RVAL(code) ((code) & 0x18)
#define BPF_A 0x10
#define BPF_TAX 0x00
#define BPF_TXA 0x80
-#ifndef BPF_MAXINSNS
-#define BPF_MAXINSNS 4096
-#endif
-
/*
* Macros for filter block array initializers.
*/
#define RAID_VERSION _IOR (MD_MAJOR, 0x10, mdu_version_t)
#define GET_ARRAY_INFO _IOR (MD_MAJOR, 0x11, mdu_array_info_t)
#define GET_DISK_INFO _IOR (MD_MAJOR, 0x12, mdu_disk_info_t)
-#define PRINT_RAID_DEBUG _IO (MD_MAJOR, 0x13)
#define RAID_AUTORUN _IO (MD_MAJOR, 0x14)
#define GET_BITMAP_FILE _IOR (MD_MAJOR, 0x15, mdu_bitmap_file_t)
case FUT_OFF_MMSHARED:
futex_get_mm(key); /* implies MB (B) */
break;
+ default:
+ smp_mb(); /* explicit MB (B) */
}
}
/* We leave it in list to prevent duplicate loads, but make sure
* that noone uses it while it's being deconstructed. */
+ mutex_lock(&module_mutex);
mod->state = MODULE_STATE_UNFORMED;
+ mutex_unlock(&module_mutex);
/* Remove dynamic debug info */
ddebug_remove_module(mod->name);
raid6_2data_recov = best->data2;
raid6_datap_recov = best->datap;
- printk("raid6: using %s recovery algorithm\n", best->name);
+ pr_info("raid6: using %s recovery algorithm\n", best->name);
} else
- printk("raid6: Yikes! No recovery algorithm found!\n");
+ pr_err("raid6: Yikes! No recovery algorithm found!\n");
return best;
}
bestperf = perf;
best = *algo;
}
- printk("raid6: %-8s %5ld MB/s\n", (*algo)->name,
+ pr_info("raid6: %-8s %5ld MB/s\n", (*algo)->name,
(perf*HZ) >> (20-16+RAID6_TIME_JIFFIES_LG2));
}
}
if (best) {
- printk("raid6: using algorithm %s (%ld MB/s)\n",
+ pr_info("raid6: using algorithm %s (%ld MB/s)\n",
best->name,
(bestperf*HZ) >> (20-16+RAID6_TIME_JIFFIES_LG2));
raid6_call = *best;
} else
- printk("raid6: Yikes! No algorithm found!\n");
+ pr_err("raid6: Yikes! No algorithm found!\n");
return best;
}
syndromes = (void *) __get_free_pages(GFP_KERNEL, 1);
if (!syndromes) {
- printk("raid6: Yikes! No memory available.\n");
+ pr_err("raid6: Yikes! No memory available.\n");
return -ENOMEM;
}
/* bdi_wq serves all asynchronous writeback tasks */
struct workqueue_struct *bdi_wq;
-void bdi_lock_two(struct bdi_writeback *wb1, struct bdi_writeback *wb2)
+static void bdi_lock_two(struct bdi_writeback *wb1, struct bdi_writeback *wb2)
{
if (wb1 < wb2) {
spin_lock(&wb1->list_lock);
mod_delayed_work(bdi_wq, &bdi->wb.dwork, 0);
flush_delayed_work(&bdi->wb.dwork);
WARN_ON(!list_empty(&bdi->work_list));
-
- /*
- * This shouldn't be necessary unless @bdi for some reason has
- * unflushed dirty IO after work_list is drained. Do it anyway
- * just in case.
- */
- cancel_delayed_work_sync(&bdi->wb.dwork);
+ WARN_ON(delayed_work_pending(&bdi->wb.dwork));
}
/*
void bdi_unregister(struct backing_dev_info *bdi)
{
- struct device *dev = bdi->dev;
-
- if (dev) {
+ if (bdi->dev) {
bdi_set_min_ratio(bdi, 0);
trace_writeback_bdi_unregister(bdi);
bdi_prune_sb(bdi);
bdi_wb_shutdown(bdi);
bdi_debug_unregister(bdi);
-
- spin_lock_bh(&bdi->wb_lock);
+ device_unregister(bdi->dev);
bdi->dev = NULL;
- spin_unlock_bh(&bdi->wb_lock);
-
- device_unregister(dev);
}
}
EXPORT_SYMBOL(bdi_unregister);
int i;
/*
- * Splice our entries to the default_backing_dev_info, if this
- * bdi disappears
+ * Splice our entries to the default_backing_dev_info. This
+ * condition shouldn't happen. @wb must be empty at this point and
+ * dirty inodes on it might cause other issues. This workaround is
+ * added by ce5f8e779519 ("writeback: splice dirty inode entries to
+ * default bdi on bdi_destroy()") without root-causing the issue.
+ *
+ * http://lkml.kernel.org/g/1253038617-30204-11-git-send-email-jens.axboe@oracle.com
+ * http://thread.gmane.org/gmane.linux.file-systems/35341/focus=35350
+ *
+ * We should probably add WARN_ON() to find out whether it still
+ * happens and track it down if so.
*/
if (bdi_has_dirty_io(bdi)) {
struct bdi_writeback *dst = &default_backing_dev_info.wb;
bdi_unregister(bdi);
- /*
- * If bdi_unregister() had already been called earlier, the dwork
- * could still be pending because bdi_prune_sb() can race with the
- * bdi_wakeup_thread_delayed() calls from __mark_inode_dirty().
- */
- cancel_delayed_work_sync(&bdi->wb.dwork);
+ WARN_ON(delayed_work_pending(&bdi->wb.dwork));
for (i = 0; i < NR_BDI_STAT_ITEMS; i++)
percpu_counter_destroy(&bdi->bdi_stat[i]);
/* Ceiling limit to avoid denial of service attacks */
chan->p9_max_pages = nr_free_buffer_pages()/4;
+ virtio_device_ready(vdev);
+
mutex_lock(&virtio_9p_lock);
list_add_tail(&chan->chan_list, &virtio_chan_list);
mutex_unlock(&virtio_9p_lock);
{
struct cfusbl *this = kmalloc(sizeof(struct cfusbl), GFP_ATOMIC);
- if (!this) {
- pr_warn("Out of memory\n");
+ if (!this)
return NULL;
- }
+
caif_assert(offsetof(struct cfusbl, layer) == 0);
- memset(this, 0, sizeof(struct cflayer));
+ memset(&this->layer, 0, sizeof(this->layer));
this->layer.receive = cfusbl_receive;
this->layer.transmit = cfusbl_transmit;
this->layer.ctrlcmd = cfusbl_ctrlcmd;
struct cflayer *cfmuxl_create(void)
{
- struct cfmuxl *this = kmalloc(sizeof(struct cfmuxl), GFP_ATOMIC);
+ struct cfmuxl *this = kzalloc(sizeof(struct cfmuxl), GFP_ATOMIC);
+
if (!this)
return NULL;
- memset(this, 0, sizeof(*this));
this->layer.receive = cfmuxl_receive;
this->layer.transmit = cfmuxl_transmit;
this->layer.ctrlcmd = cfmuxl_ctrlcmd;
if (skb->encapsulation)
features &= dev->hw_enc_features;
- if (netif_needs_gso(skb, features)) {
+ if (netif_needs_gso(dev, skb, features)) {
struct sk_buff *segs;
segs = skb_gso_segment(skb, features);
*/
void sock_kfree_s(struct sock *sk, void *mem, int size)
{
+ if (WARN_ON_ONCE(!mem))
+ return;
kfree(mem);
atomic_sub(size, &sk->sk_omem_alloc);
}
/* Step 2:
* Look up flow ID in table and get corresponding socket */
sk = __inet6_lookup_skb(&dccp_hashinfo, skb,
- dh->dccph_sport, dh->dccph_dport);
+ dh->dccph_sport, dh->dccph_dport,
+ inet6_iif(skb));
/*
* Step 2:
* If no socket ...
#include <linux/list.h>
#include <linux/etherdevice.h>
#include <linux/phy.h>
+#include <linux/phy_fixed.h>
#include <linux/of_net.h>
#include <linux/of_mdio.h>
#include "dsa_priv.h"
return 1;
attrlen = rtnh_attrlen(rtnh);
- if (attrlen < 0) {
+ if (attrlen > 0) {
struct nlattr *nla, *attrs = rtnh_attrs(rtnh);
nla = nla_find(attrs, attrlen, RTA_GATEWAY);
if (!pskb_may_pull(skb, len))
goto drop;
+ uh = udp_hdr(skb);
+ guehdr = (struct guehdr *)&uh[1];
+
if (guehdr->version != 0)
goto drop;
struct sk_buff *nskb;
struct sock *sk;
struct inet_sock *inet;
+ int err;
if (__ip_options_echo(&replyopts.opt.opt, skb, sopt))
return;
sock_net_set(sk, net);
__skb_queue_head_init(&sk->sk_write_queue);
sk->sk_sndbuf = sysctl_wmem_default;
- ip_append_data(sk, &fl4, ip_reply_glue_bits, arg->iov->iov_base, len, 0,
- &ipc, &rt, MSG_DONTWAIT);
+ err = ip_append_data(sk, &fl4, ip_reply_glue_bits, arg->iov->iov_base,
+ len, 0, &ipc, &rt, MSG_DONTWAIT);
+ if (unlikely(err)) {
+ ip_flush_pending_frames(sk);
+ goto out;
+ }
+
nskb = skb_peek(&sk->sk_write_queue);
if (nskb) {
if (arg->csumoffset >= 0)
skb_set_queue_mapping(nskb, skb_get_queue_mapping(skb));
ip_push_pending_frames(sk, &fl4);
}
-
+out:
put_cpu_var(unicast_sock);
ip_rt_put(rt);
skb_pull_rcsum(skb, hdr_len);
if (inner_proto == htons(ETH_P_TEB)) {
- struct ethhdr *eh = (struct ethhdr *)skb->data;
+ struct ethhdr *eh;
if (unlikely(!pskb_may_pull(skb, ETH_HLEN)))
return -ENOMEM;
+ eh = (struct ethhdr *)skb->data;
if (likely(ntohs(eh->h_proto) >= ETH_P_802_3_MIN))
skb->protocol = eh->h_proto;
else
}
EXPORT_SYMBOL(cookie_check_timestamp);
-struct sock *cookie_v4_check(struct sock *sk, struct sk_buff *skb,
- struct ip_options *opt)
+struct sock *cookie_v4_check(struct sock *sk, struct sk_buff *skb)
{
+ struct ip_options *opt = &TCP_SKB_CB(skb)->header.h4.opt;
struct tcp_options_received tcp_opt;
struct inet_request_sock *ireq;
struct tcp_request_sock *treq;
/* We throwed the options of the initial SYN away, so we hope
* the ACK carries the same options again (see RFC1122 4.2.3.8)
*/
- if (opt && opt->optlen) {
- int opt_size = sizeof(struct ip_options_rcu) + opt->optlen;
-
- ireq->opt = kmalloc(opt_size, GFP_ATOMIC);
- if (ireq->opt != NULL && ip_options_echo(&ireq->opt->opt, skb)) {
- kfree(ireq->opt);
- ireq->opt = NULL;
- }
- }
+ ireq->opt = tcp_v4_save_options(skb);
if (security_inet_conn_request(sk, skb, req)) {
reqsk_free(req);
flowi4_init_output(&fl4, sk->sk_bound_dev_if, ireq->ir_mark,
RT_CONN_FLAGS(sk), RT_SCOPE_UNIVERSE, IPPROTO_TCP,
inet_sk_flowi_flags(sk),
- (opt && opt->srr) ? opt->faddr : ireq->ir_rmt_addr,
+ opt->srr ? opt->faddr : ireq->ir_rmt_addr,
ireq->ir_loc_addr, th->source, th->dest);
security_req_classify_flow(req, flowi4_to_flowi(&fl4));
rt = ip_route_output_key(sock_net(sk), &fl4);
#include <linux/module.h>
#include <linux/sysctl.h>
#include <linux/kernel.h>
+#include <linux/prefetch.h>
#include <net/dst.h>
#include <net/tcp.h>
#include <net/inet_common.h>
return packets_acked;
}
+static void tcp_ack_tstamp(struct sock *sk, struct sk_buff *skb,
+ u32 prior_snd_una)
+{
+ const struct skb_shared_info *shinfo;
+
+ /* Avoid cache line misses to get skb_shinfo() and shinfo->tx_flags */
+ if (likely(!(sk->sk_tsflags & SOF_TIMESTAMPING_TX_ACK)))
+ return;
+
+ shinfo = skb_shinfo(skb);
+ if ((shinfo->tx_flags & SKBTX_ACK_TSTAMP) &&
+ between(shinfo->tskey, prior_snd_una, tcp_sk(sk)->snd_una - 1))
+ __skb_tstamp_tx(skb, NULL, sk, SCM_TSTAMP_ACK);
+}
+
/* Remove acknowledged frames from the retransmission queue. If our packet
* is before the ack sequence we can discard it as it's confirmed to have
* arrived at the other end.
first_ackt.v64 = 0;
while ((skb = tcp_write_queue_head(sk)) && skb != tcp_send_head(sk)) {
- struct skb_shared_info *shinfo = skb_shinfo(skb);
struct tcp_skb_cb *scb = TCP_SKB_CB(skb);
u8 sacked = scb->sacked;
u32 acked_pcount;
- if (unlikely(shinfo->tx_flags & SKBTX_ACK_TSTAMP) &&
- between(shinfo->tskey, prior_snd_una, tp->snd_una - 1))
- __skb_tstamp_tx(skb, NULL, sk, SCM_TSTAMP_ACK);
+ tcp_ack_tstamp(sk, skb, prior_snd_una);
/* Determine how many packets and what bytes were acked, tso and else */
if (after(scb->end_seq, tp->snd_una)) {
fully_acked = false;
} else {
+ /* Speedup tcp_unlink_write_queue() and next loop */
+ prefetchw(skb->next);
acked_pcount = tcp_skb_pcount(skb);
}
- if (sacked & TCPCB_RETRANS) {
+ if (unlikely(sacked & TCPCB_RETRANS)) {
if (sacked & TCPCB_SACKED_RETRANS)
tp->retrans_out -= acked_pcount;
flag |= FLAG_RETRANS_DATA_ACKED;
* connection startup slow start one packet too
* quickly. This is severely frowned upon behavior.
*/
- if (!(scb->tcp_flags & TCPHDR_SYN)) {
+ if (likely(!(scb->tcp_flags & TCPHDR_SYN))) {
flag |= FLAG_DATA_ACKED;
} else {
flag |= FLAG_SYN_ACKED;
tcp_unlink_write_queue(skb, sk);
sk_wmem_free_skb(sk, skb);
- if (skb == tp->retransmit_skb_hint)
+ if (unlikely(skb == tp->retransmit_skb_hint))
tp->retransmit_skb_hint = NULL;
- if (skb == tp->lost_skb_hint)
+ if (unlikely(skb == tp->lost_skb_hint))
tp->lost_skb_hint = NULL;
}
flag |= FLAG_SACK_RENEGING;
skb_mstamp_get(&now);
- if (first_ackt.v64) {
+ if (likely(first_ackt.v64)) {
seq_rtt_us = skb_mstamp_us_delta(&now, &first_ackt);
ca_seq_rtt_us = skb_mstamp_us_delta(&now, &last_ackt);
}
int acked = 0; /* Number of packets newly acked */
long sack_rtt_us = -1L;
+ /* We very likely will need to access write queue head. */
+ prefetchw(sk->sk_write_queue.next);
+
/* If the ack is older than previous acks
* then we can probably ignore it.
*/
}
EXPORT_SYMBOL(tcp_syn_flood_action);
-/*
- * Save and compile IPv4 options into the request_sock if needed.
- */
-static struct ip_options_rcu *tcp_v4_save_options(struct sk_buff *skb)
-{
- const struct ip_options *opt = &TCP_SKB_CB(skb)->header.h4.opt;
- struct ip_options_rcu *dopt = NULL;
-
- if (opt && opt->optlen) {
- int opt_size = sizeof(*dopt) + opt->optlen;
-
- dopt = kmalloc(opt_size, GFP_ATOMIC);
- if (dopt && __ip_options_echo(&dopt->opt, skb, opt)) {
- kfree(dopt);
- dopt = NULL;
- }
- }
- return dopt;
-}
-
#ifdef CONFIG_TCP_MD5SIG
/*
* RFC2385 MD5 checksumming requires a mapping of
#ifdef CONFIG_SYN_COOKIES
if (!th->syn)
- sk = cookie_v4_check(sk, skb, &TCP_SKB_CB(skb)->header.h4.opt);
+ sk = cookie_v4_check(sk, skb);
#endif
return sk;
}
{
struct sock *sk = skb->sk;
struct tcp_sock *tp = tcp_sk(sk);
+ int wmem;
+
+ /* Keep one reference on sk_wmem_alloc.
+ * Will be released by sk_free() from here or tcp_tasklet_func()
+ */
+ wmem = atomic_sub_return(skb->truesize - 1, &sk->sk_wmem_alloc);
+
+ /* If this softirq is serviced by ksoftirqd, we are likely under stress.
+ * Wait until our queues (qdisc + devices) are drained.
+ * This gives :
+ * - less callbacks to tcp_write_xmit(), reducing stress (batches)
+ * - chance for incoming ACK (processed by another cpu maybe)
+ * to migrate this flow (skb->ooo_okay will be eventually set)
+ */
+ if (wmem >= SKB_TRUESIZE(1) && this_cpu_ksoftirqd() == current)
+ goto out;
if (test_and_clear_bit(TSQ_THROTTLED, &tp->tsq_flags) &&
!test_and_set_bit(TSQ_QUEUED, &tp->tsq_flags)) {
unsigned long flags;
struct tsq_tasklet *tsq;
- /* Keep a ref on socket.
- * This last ref will be released in tcp_tasklet_func()
- */
- atomic_sub(skb->truesize - 1, &sk->sk_wmem_alloc);
-
/* queue this socket to tasklet queue */
local_irq_save(flags);
tsq = this_cpu_ptr(&tsq_tasklet);
list_add(&tp->tsq_node, &tsq->head);
tasklet_schedule(&tsq->tasklet);
local_irq_restore(flags);
- } else {
- sock_wfree(skb);
+ return;
}
+out:
+ sk_free(sk);
}
/* This routine actually transmits TCP packets queued in by
tcp_ca_event(sk, CA_EVENT_TX_START);
/* if no packet is in qdisc/device queue, then allow XPS to select
- * another queue.
+ * another queue. We can be called from tcp_tsq_handler()
+ * which holds one reference to sk_wmem_alloc.
+ *
+ * TODO: Ideally, in-flight pure ACK packets should not matter here.
+ * One way to get this would be to set skb->truesize = 2 on them.
*/
- skb->ooo_okay = sk_wmem_alloc_get(sk) == 0;
+ skb->ooo_okay = sk_wmem_alloc_get(sk) < SKB_TRUESIZE(1);
skb_push(skb, tcp_header_size);
skb_reset_transport_header(skb);
/* aca_tstamp should be updated upon changes */
aca->aca_cstamp = aca->aca_tstamp = jiffies;
atomic_set(&aca->aca_refcnt, 1);
- spin_lock_init(&aca->aca_lock);
return aca;
}
/* So that link locals have meaning */
if (!sk->sk_bound_dev_if &&
ipv6_addr_type(&ireq->ir_v6_rmt_addr) & IPV6_ADDR_LINKLOCAL)
- ireq->ir_iif = inet6_iif(skb);
+ ireq->ir_iif = tcp_v6_iif(skb);
ireq->ir_mark = inet_request_mark(sk, skb);
if (sock_owned_by_user(sk))
goto out;
+ /* Note : We use inet6_iif() here, not tcp_v6_iif() */
req = inet6_csk_search_req(sk, &prev, th->dest, &hdr->daddr,
&hdr->saddr, inet6_iif(skb));
if (!req)
/* So that link locals have meaning */
if (!sk->sk_bound_dev_if &&
ipv6_addr_type(&ireq->ir_v6_rmt_addr) & IPV6_ADDR_LINKLOCAL)
- ireq->ir_iif = inet6_iif(skb);
+ ireq->ir_iif = tcp_v6_iif(skb);
if (!TCP_SKB_CB(skb)->tcp_tw_isn &&
(ipv6_opt_accepted(sk, skb, &TCP_SKB_CB(skb)->header.h6) ||
fl6.flowi6_proto = IPPROTO_TCP;
if (rt6_need_strict(&fl6.daddr) && !oif)
- fl6.flowi6_oif = inet6_iif(skb);
+ fl6.flowi6_oif = tcp_v6_iif(skb);
else
fl6.flowi6_oif = oif;
fl6.flowi6_mark = IP6_REPLY_MARK(net, skb->mark);
sk1 = inet6_lookup_listener(dev_net(skb_dst(skb)->dev),
&tcp_hashinfo, &ipv6h->saddr,
th->source, &ipv6h->daddr,
- ntohs(th->source), inet6_iif(skb));
+ ntohs(th->source), tcp_v6_iif(skb));
if (!sk1)
return;
/* Find possible connection requests. */
req = inet6_csk_search_req(sk, &prev, th->source,
&ipv6_hdr(skb)->saddr,
- &ipv6_hdr(skb)->daddr, inet6_iif(skb));
+ &ipv6_hdr(skb)->daddr, tcp_v6_iif(skb));
if (req)
return tcp_check_req(sk, skb, req, prev, false);
nsk = __inet6_lookup_established(sock_net(sk), &tcp_hashinfo,
- &ipv6_hdr(skb)->saddr, th->source,
- &ipv6_hdr(skb)->daddr, ntohs(th->dest), inet6_iif(skb));
+ &ipv6_hdr(skb)->saddr, th->source,
+ &ipv6_hdr(skb)->daddr, ntohs(th->dest),
+ tcp_v6_iif(skb));
if (nsk) {
if (nsk->sk_state != TCP_TIME_WAIT) {
newnp->ipv6_fl_list = NULL;
newnp->pktoptions = NULL;
newnp->opt = NULL;
- newnp->mcast_oif = inet6_iif(skb);
+ newnp->mcast_oif = tcp_v6_iif(skb);
newnp->mcast_hops = ipv6_hdr(skb)->hop_limit;
newnp->rcv_flowinfo = ip6_flowinfo(ipv6_hdr(skb));
if (np->repflow)
skb_set_owner_r(newnp->pktoptions, newsk);
}
newnp->opt = NULL;
- newnp->mcast_oif = inet6_iif(skb);
+ newnp->mcast_oif = tcp_v6_iif(skb);
newnp->mcast_hops = ipv6_hdr(skb)->hop_limit;
newnp->rcv_flowinfo = ip6_flowinfo(ipv6_hdr(skb));
if (np->repflow)
if (TCP_SKB_CB(opt_skb)->end_seq == tp->rcv_nxt &&
!((1 << sk->sk_state) & (TCPF_CLOSE | TCPF_LISTEN))) {
if (np->rxopt.bits.rxinfo || np->rxopt.bits.rxoinfo)
- np->mcast_oif = inet6_iif(opt_skb);
+ np->mcast_oif = tcp_v6_iif(opt_skb);
if (np->rxopt.bits.rxhlim || np->rxopt.bits.rxohlim)
np->mcast_hops = ipv6_hdr(opt_skb)->hop_limit;
if (np->rxopt.bits.rxflow || np->rxopt.bits.rxtclass)
TCP_SKB_CB(skb)->ip_dsfield = ipv6_get_dsfield(hdr);
TCP_SKB_CB(skb)->sacked = 0;
- sk = __inet6_lookup_skb(&tcp_hashinfo, skb, th->source, th->dest);
+ sk = __inet6_lookup_skb(&tcp_hashinfo, skb, th->source, th->dest,
+ tcp_v6_iif(skb));
if (!sk)
goto no_tcp_socket;
sk2 = inet6_lookup_listener(dev_net(skb->dev), &tcp_hashinfo,
&ipv6_hdr(skb)->saddr, th->source,
&ipv6_hdr(skb)->daddr,
- ntohs(th->dest), inet6_iif(skb));
+ ntohs(th->dest), tcp_v6_iif(skb));
if (sk2 != NULL) {
struct inet_timewait_sock *tw = inet_twsk(sk);
inet_twsk_deschedule(tw, &tcp_death_row);
if (th->doff < sizeof(struct tcphdr) / 4)
return;
+ /* Note : We use inet6_iif() here, not tcp_v6_iif() */
sk = __inet6_lookup_established(dev_net(skb->dev), &tcp_hashinfo,
&hdr->saddr, th->source,
&hdr->daddr, ntohs(th->dest),
#include <linux/skbuff.h>
#include <net/net_namespace.h>
#include <net/sock.h>
-#include <asm/uaccess.h>
+#include <linux/uaccess.h>
#include <linux/fcntl.h>
#include <linux/termios.h> /* For TIOCINQ/OUTQ */
#include <linux/mm.h>
#include <linux/in.h>
#include <linux/if_ether.h> /* For the statistics structure. */
#include <linux/slab.h>
+#include <linux/uaccess.h>
-#include <asm/uaccess.h>
#include <asm/io.h>
#include <linux/inet.h>
#include <linux/skbuff.h>
#include <net/sock.h>
#include <net/tcp_states.h>
-#include <asm/uaccess.h>
+#include <linux/uaccess.h>
#include <linux/fcntl.h>
#include <linux/mm.h>
#include <linux/interrupt.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
#include <net/sock.h>
-#include <asm/uaccess.h>
+#include <linux/uaccess.h>
#include <linux/fcntl.h>
#include <linux/mm.h>
#include <linux/interrupt.h>
#include <linux/if_arp.h>
#include <linux/skbuff.h>
#include <net/sock.h>
-#include <asm/uaccess.h>
+#include <linux/uaccess.h>
#include <linux/fcntl.h>
#include <linux/termios.h> /* For TIOCINQ/OUTQ */
#include <linux/mm.h>
#include <linux/skbuff.h>
#include <net/sock.h>
#include <net/tcp_states.h>
-#include <asm/uaccess.h>
+#include <linux/uaccess.h>
#include <linux/fcntl.h>
#include <linux/mm.h>
#include <linux/interrupt.h>
#include <linux/skbuff.h>
#include <net/sock.h>
#include <net/tcp_states.h>
-#include <asm/uaccess.h>
+#include <linux/uaccess.h>
#include <linux/fcntl.h>
#include <linux/mm.h>
#include <linux/interrupt.h>
key->ip.frag = OVS_FRAG_TYPE_LATER;
else
key->ip.frag = OVS_FRAG_TYPE_FIRST;
+ } else {
+ key->ip.frag = OVS_FRAG_TYPE_NONE;
}
nh_len = payload_ofs - nh_ofs;
*/
key->tp.src = htons(icmp->icmp6_type);
key->tp.dst = htons(icmp->icmp6_code);
+ memset(&key->ipv6.nd, 0, sizeof(key->ipv6.nd));
if (icmp->icmp6_code == 0 &&
(icmp->icmp6_type == NDISC_NEIGHBOUR_SOLICITATION ||
} else if (key->eth.type == htons(ETH_P_ARP) ||
key->eth.type == htons(ETH_P_RARP)) {
struct arp_eth_header *arp;
+ bool arp_available = arphdr_ok(skb);
arp = (struct arp_eth_header *)skb_network_header(skb);
- if (arphdr_ok(skb) &&
+ if (arp_available &&
arp->ar_hrd == htons(ARPHRD_ETHER) &&
arp->ar_pro == htons(ETH_P_IP) &&
arp->ar_hln == ETH_ALEN &&
key->ovs_flow_hash = 0;
key->recirc_id = 0;
- /* Flags are always used as part of stats */
- key->tp.flags = 0;
-
return key_extract(skb, key);
}
SW_FLOW_KEY_MEMCPY_OFFSET(match, offsetof(struct sw_flow_key, field), \
value_p, len, is_mask)
-static u16 range_n_bytes(const struct sw_flow_key_range *range)
-{
- return range->end - range->start;
-}
+#define SW_FLOW_KEY_MEMSET_FIELD(match, field, value, is_mask) \
+ do { \
+ update_range__(match, offsetof(struct sw_flow_key, field), \
+ sizeof((match)->key->field), is_mask); \
+ if (is_mask) { \
+ if ((match)->mask) \
+ memset((u8 *)&(match)->mask->key.field, value,\
+ sizeof((match)->mask->key.field)); \
+ } else { \
+ memset((u8 *)&(match)->key->field, value, \
+ sizeof((match)->key->field)); \
+ } \
+ } while (0)
static bool match_validate(const struct sw_flow_match *match,
u64 key_attrs, u64 mask_attrs)
return 0;
}
-static void sw_flow_mask_set(struct sw_flow_mask *mask,
- struct sw_flow_key_range *range, u8 val)
+static void nlattr_set(struct nlattr *attr, u8 val, bool is_attr_mask_key)
{
- u8 *m = (u8 *)&mask->key + range->start;
+ struct nlattr *nla;
+ int rem;
+
+ /* The nlattr stream should already have been validated */
+ nla_for_each_nested(nla, attr, rem) {
+ /* We assume that ovs_key_lens[type] == -1 means that type is a
+ * nested attribute
+ */
+ if (is_attr_mask_key && ovs_key_lens[nla_type(nla)] == -1)
+ nlattr_set(nla, val, false);
+ else
+ memset(nla_data(nla), val, nla_len(nla));
+ }
+}
- mask->range = *range;
- memset(m, val, range_n_bytes(range));
+static void mask_set_nlattr(struct nlattr *attr, u8 val)
+{
+ nlattr_set(attr, val, true);
}
/**
{
const struct nlattr *a[OVS_KEY_ATTR_MAX + 1];
const struct nlattr *encap;
+ struct nlattr *newmask = NULL;
u64 key_attrs = 0;
u64 mask_attrs = 0;
bool encap_valid = false;
if (err)
return err;
+ if (match->mask && !mask) {
+ /* Create an exact match mask. We need to set to 0xff all the
+ * 'match->mask' fields that have been touched in 'match->key'.
+ * We cannot simply memset 'match->mask', because padding bytes
+ * and fields not specified in 'match->key' should be left to 0.
+ * Instead, we use a stream of netlink attributes, copied from
+ * 'key' and set to 0xff: ovs_key_from_nlattrs() will take care
+ * of filling 'match->mask' appropriately.
+ */
+ newmask = kmemdup(key, nla_total_size(nla_len(key)),
+ GFP_KERNEL);
+ if (!newmask)
+ return -ENOMEM;
+
+ mask_set_nlattr(newmask, 0xff);
+
+ /* The userspace does not send tunnel attributes that are 0,
+ * but we should not wildcard them nonetheless.
+ */
+ if (match->key->tun_key.ipv4_dst)
+ SW_FLOW_KEY_MEMSET_FIELD(match, tun_key, 0xff, true);
+
+ mask = newmask;
+ }
+
if (mask) {
err = parse_flow_mask_nlattrs(mask, a, &mask_attrs);
if (err)
- return err;
+ goto free_newmask;
- if (mask_attrs & 1 << OVS_KEY_ATTR_ENCAP) {
+ if (mask_attrs & 1 << OVS_KEY_ATTR_ENCAP) {
__be16 eth_type = 0;
__be16 tci = 0;
if (!encap_valid) {
OVS_NLERR("Encap mask attribute is set for non-VLAN frame.\n");
- return -EINVAL;
+ err = -EINVAL;
+ goto free_newmask;
}
mask_attrs &= ~(1 << OVS_KEY_ATTR_ENCAP);
mask_attrs &= ~(1 << OVS_KEY_ATTR_ETHERTYPE);
encap = a[OVS_KEY_ATTR_ENCAP];
err = parse_flow_mask_nlattrs(encap, a, &mask_attrs);
+ if (err)
+ goto free_newmask;
} else {
OVS_NLERR("VLAN frames must have an exact match on the TPID (mask=%x).\n",
ntohs(eth_type));
- return -EINVAL;
+ err = -EINVAL;
+ goto free_newmask;
}
if (a[OVS_KEY_ATTR_VLAN])
if (!(tci & htons(VLAN_TAG_PRESENT))) {
OVS_NLERR("VLAN tag present bit must have an exact match (tci_mask=%x).\n", ntohs(tci));
- return -EINVAL;
+ err = -EINVAL;
+ goto free_newmask;
}
}
err = ovs_key_from_nlattrs(match, mask_attrs, a, true);
if (err)
- return err;
- } else {
- /* Populate exact match flow's key mask. */
- if (match->mask)
- sw_flow_mask_set(match->mask, &match->range, 0xff);
+ goto free_newmask;
}
if (!match_validate(match, key_attrs, mask_attrs))
- return -EINVAL;
+ err = -EINVAL;
- return 0;
+free_newmask:
+ kfree(newmask);
+ return err;
}
/**
/**
* struct geneve_port - Keeps track of open UDP ports
- * @sock: The socket created for this port number.
+ * @gs: The socket created for this port number.
* @name: vport name.
*/
struct geneve_port {
*
* Returns the portid of the target socket. Must be called with rcu_read_lock.
*/
-u32 ovs_vport_find_upcall_portid(const struct vport *p, struct sk_buff *skb)
+u32 ovs_vport_find_upcall_portid(const struct vport *vport, struct sk_buff *skb)
{
struct vport_portids *ids;
u32 ids_index;
u32 hash;
- ids = rcu_dereference(p->upcall_portids);
+ ids = rcu_dereference(vport->upcall_portids);
if (ids->n_ids == 1 && ids->ids[0] == 0)
return 0;
if (rs->rs_bound_addr == 0) {
ret = -ENOTCONN; /* XXX not a great errno */
- goto out;
+ goto out_ret;
}
if (args->nr_local > UIO_MAXIOV) {
ret = -EMSGSIZE;
- goto out;
+ goto out_ret;
}
/* Check whether to allocate the iovec area */
iovs = sock_kmalloc(rds_rs_to_sk(rs), iov_size, GFP_KERNEL);
if (!iovs) {
ret = -ENOMEM;
- goto out;
+ goto out_ret;
}
}
if (iovs != iovstack)
sock_kfree_s(rds_rs_to_sk(rs), iovs, iov_size);
kfree(pages);
+out_ret:
if (ret)
rds_rdma_free_op(op);
else
* ack chunk whose serial number matches that of the request.
*/
list_for_each_entry(ack, &asoc->asconf_ack_list, transmitted_list) {
+ if (sctp_chunk_pending(ack))
+ continue;
if (ack->subh.addip_hdr->serial == serial) {
sctp_chunk_hold(ack);
return ack;
} else {
/* Nothing to do. Next chunk in the packet, please. */
ch = (sctp_chunkhdr_t *) chunk->chunk_end;
-
/* Force chunk->skb->data to chunk->chunk_end. */
- skb_pull(chunk->skb,
- chunk->chunk_end - chunk->skb->data);
-
- /* Verify that we have at least chunk headers
- * worth of buffer left.
- */
- if (skb_headlen(chunk->skb) < sizeof(sctp_chunkhdr_t)) {
- sctp_chunk_free(chunk);
- chunk = queue->in_progress = NULL;
- }
+ skb_pull(chunk->skb, chunk->chunk_end - chunk->skb->data);
+ /* We are guaranteed to pull a SCTP header. */
}
}
skb_pull(chunk->skb, sizeof(sctp_chunkhdr_t));
chunk->subh.v = NULL; /* Subheader is no longer valid. */
- if (chunk->chunk_end < skb_tail_pointer(chunk->skb)) {
+ if (chunk->chunk_end + sizeof(sctp_chunkhdr_t) <
+ skb_tail_pointer(chunk->skb)) {
/* This is not a singleton */
chunk->singleton = 0;
} else if (chunk->chunk_end > skb_tail_pointer(chunk->skb)) {
- /* RFC 2960, Section 6.10 Bundling
- *
- * Partial chunks MUST NOT be placed in an SCTP packet.
- * If the receiver detects a partial chunk, it MUST drop
- * the chunk.
- *
- * Since the end of the chunk is past the end of our buffer
- * (which contains the whole packet, we can freely discard
- * the whole packet.
- */
- sctp_chunk_free(chunk);
- chunk = queue->in_progress = NULL;
-
- return NULL;
+ /* Discard inside state machine. */
+ chunk->pdiscard = 1;
+ chunk->chunk_end = skb_tail_pointer(chunk->skb);
} else {
/* We are at the end of the packet, so mark the chunk
* in case we need to send a SACK.
return SCTP_ERROR_NO_ERROR;
}
-/* Verify the ASCONF packet before we process it. */
-int sctp_verify_asconf(const struct sctp_association *asoc,
- struct sctp_paramhdr *param_hdr, void *chunk_end,
- struct sctp_paramhdr **errp) {
- sctp_addip_param_t *asconf_param;
+/* Verify the ASCONF packet before we process it. */
+bool sctp_verify_asconf(const struct sctp_association *asoc,
+ struct sctp_chunk *chunk, bool addr_param_needed,
+ struct sctp_paramhdr **errp)
+{
+ sctp_addip_chunk_t *addip = (sctp_addip_chunk_t *) chunk->chunk_hdr;
union sctp_params param;
- int length, plen;
-
- param.v = (sctp_paramhdr_t *) param_hdr;
- while (param.v <= chunk_end - sizeof(sctp_paramhdr_t)) {
- length = ntohs(param.p->length);
- *errp = param.p;
+ bool addr_param_seen = false;
- if (param.v > chunk_end - length ||
- length < sizeof(sctp_paramhdr_t))
- return 0;
+ sctp_walk_params(param, addip, addip_hdr.params) {
+ size_t length = ntohs(param.p->length);
+ *errp = param.p;
switch (param.p->type) {
+ case SCTP_PARAM_ERR_CAUSE:
+ break;
+ case SCTP_PARAM_IPV4_ADDRESS:
+ if (length != sizeof(sctp_ipv4addr_param_t))
+ return false;
+ addr_param_seen = true;
+ break;
+ case SCTP_PARAM_IPV6_ADDRESS:
+ if (length != sizeof(sctp_ipv6addr_param_t))
+ return false;
+ addr_param_seen = true;
+ break;
case SCTP_PARAM_ADD_IP:
case SCTP_PARAM_DEL_IP:
case SCTP_PARAM_SET_PRIMARY:
- asconf_param = (sctp_addip_param_t *)param.v;
- plen = ntohs(asconf_param->param_hdr.length);
- if (plen < sizeof(sctp_addip_param_t) +
- sizeof(sctp_paramhdr_t))
- return 0;
+ /* In ASCONF chunks, these need to be first. */
+ if (addr_param_needed && !addr_param_seen)
+ return false;
+ length = ntohs(param.addip->param_hdr.length);
+ if (length < sizeof(sctp_addip_param_t) +
+ sizeof(sctp_paramhdr_t))
+ return false;
break;
case SCTP_PARAM_SUCCESS_REPORT:
case SCTP_PARAM_ADAPTATION_LAYER_IND:
if (length != sizeof(sctp_addip_param_t))
- return 0;
-
+ return false;
break;
default:
- break;
+ /* This is unkown to us, reject! */
+ return false;
}
-
- param.v += WORD_ROUND(length);
}
- if (param.v != chunk_end)
- return 0;
+ /* Remaining sanity checks. */
+ if (addr_param_needed && !addr_param_seen)
+ return false;
+ if (!addr_param_needed && addr_param_seen)
+ return false;
+ if (param.v != chunk->chunk_end)
+ return false;
- return 1;
+ return true;
}
/* Process an incoming ASCONF chunk with the next expected serial no. and
struct sctp_chunk *sctp_process_asconf(struct sctp_association *asoc,
struct sctp_chunk *asconf)
{
+ sctp_addip_chunk_t *addip = (sctp_addip_chunk_t *) asconf->chunk_hdr;
+ bool all_param_pass = true;
+ union sctp_params param;
sctp_addiphdr_t *hdr;
union sctp_addr_param *addr_param;
sctp_addip_param_t *asconf_param;
struct sctp_chunk *asconf_ack;
-
__be16 err_code;
int length = 0;
int chunk_len;
__u32 serial;
- int all_param_pass = 1;
chunk_len = ntohs(asconf->chunk_hdr->length) - sizeof(sctp_chunkhdr_t);
hdr = (sctp_addiphdr_t *)asconf->skb->data;
goto done;
/* Process the TLVs contained within the ASCONF chunk. */
- while (chunk_len > 0) {
+ sctp_walk_params(param, addip, addip_hdr.params) {
+ /* Skip preceeding address parameters. */
+ if (param.p->type == SCTP_PARAM_IPV4_ADDRESS ||
+ param.p->type == SCTP_PARAM_IPV6_ADDRESS)
+ continue;
+
err_code = sctp_process_asconf_param(asoc, asconf,
- asconf_param);
+ param.addip);
/* ADDIP 4.1 A7)
* If an error response is received for a TLV parameter,
* all TLVs with no response before the failed TLV are
* the failed response are considered unsuccessful unless
* a specific success indication is present for the parameter.
*/
- if (SCTP_ERROR_NO_ERROR != err_code)
- all_param_pass = 0;
-
+ if (err_code != SCTP_ERROR_NO_ERROR)
+ all_param_pass = false;
if (!all_param_pass)
- sctp_add_asconf_response(asconf_ack,
- asconf_param->crr_id, err_code,
- asconf_param);
+ sctp_add_asconf_response(asconf_ack, param.addip->crr_id,
+ err_code, param.addip);
/* ADDIP 4.3 D11) When an endpoint receiving an ASCONF to add
* an IP address sends an 'Out of Resource' in its response, it
* MUST also fail any subsequent add or delete requests bundled
* in the ASCONF.
*/
- if (SCTP_ERROR_RSRC_LOW == err_code)
+ if (err_code == SCTP_ERROR_RSRC_LOW)
goto done;
-
- /* Move to the next ASCONF param. */
- length = ntohs(asconf_param->param_hdr.length);
- asconf_param = (void *)asconf_param + length;
- chunk_len -= length;
}
-
done:
asoc->peer.addip_serial++;
{
__u16 chunk_length = ntohs(chunk->chunk_hdr->length);
+ /* Previously already marked? */
+ if (unlikely(chunk->pdiscard))
+ return 0;
if (unlikely(chunk_length < required_length))
return 0;
struct sctp_chunk *asconf_ack = NULL;
struct sctp_paramhdr *err_param = NULL;
sctp_addiphdr_t *hdr;
- union sctp_addr_param *addr_param;
__u32 serial;
- int length;
if (!sctp_vtag_verify(chunk, asoc)) {
sctp_add_cmd_sf(commands, SCTP_CMD_REPORT_BAD_TAG,
hdr = (sctp_addiphdr_t *)chunk->skb->data;
serial = ntohl(hdr->serial);
- addr_param = (union sctp_addr_param *)hdr->params;
- length = ntohs(addr_param->p.length);
- if (length < sizeof(sctp_paramhdr_t))
- return sctp_sf_violation_paramlen(net, ep, asoc, type, arg,
- (void *)addr_param, commands);
-
/* Verify the ASCONF chunk before processing it. */
- if (!sctp_verify_asconf(asoc,
- (sctp_paramhdr_t *)((void *)addr_param + length),
- (void *)chunk->chunk_end,
- &err_param))
+ if (!sctp_verify_asconf(asoc, chunk, true, &err_param))
return sctp_sf_violation_paramlen(net, ep, asoc, type, arg,
(void *)err_param, commands);
rcvd_serial = ntohl(addip_hdr->serial);
/* Verify the ASCONF-ACK chunk before processing it. */
- if (!sctp_verify_asconf(asoc,
- (sctp_paramhdr_t *)addip_hdr->params,
- (void *)asconf_ack->chunk_end,
- &err_param))
+ if (!sctp_verify_asconf(asoc, asconf_ack, false, &err_param))
return sctp_sf_violation_paramlen(net, ep, asoc, type, arg,
(void *)err_param, commands);
}
omsg = buf_msg(obuf);
pos += align(msg_size(omsg));
- if (msg_isdata(omsg) || (msg_user(omsg) == CONN_MANAGER)) {
+ if (msg_isdata(omsg)) {
+ if (unlikely(msg_type(omsg) == TIPC_MCAST_MSG))
+ tipc_sk_mcast_rcv(obuf);
+ else
+ tipc_sk_rcv(obuf);
+ } else if (msg_user(omsg) == CONN_MANAGER) {
tipc_sk_rcv(obuf);
} else if (msg_user(omsg) == NAME_DISTRIBUTOR) {
tipc_named_rcv(obuf);
list_entry(sbsec->isec_head.next,
struct inode_security_struct, list);
struct inode *inode = isec->inode;
+ list_del_init(&isec->list);
spin_unlock(&sbsec->isec_lock);
inode = igrab(inode);
if (inode) {
iput(inode);
}
spin_lock(&sbsec->isec_lock);
- list_del_init(&isec->list);
goto next_inode;
}
spin_unlock(&sbsec->isec_lock);
git.samba.org / sfrench / cifs-2.6.git / commitdiff